Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Qwen3.6-27B dense quality ladder. Same tasks. Same grader. Same day. How much does compression cost? FP8 - 83.3 · 29GB NVFP4 - 83.0 · ~15GB BF16 - 81.9 · 56GB GPTQ-Pro - 79.4 · 13GB IQ2_XXS - 79.0 · 9.4GB Who wins where: 🥇 Raw quality → FP8 (83.3)...

WescheNex1q's profile picture

Wësche

1,638 subscribers

24,599 Aufrufe • vor 3 Monaten •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

454 tok/s on Qwen3.8 27B on single RTX 6000 with +6000 mem overclock Mia’s Qwen3.8-27B NVFP4 W4A4 • SGLang + DFlash2 block-16 drafter • FP8 KV
0:19

454 tok/s on Qwen3.8 27B on single RTX 6000 with +6000 mem overclock Mia’s Qwen3.8-27B NVFP4 W4A4 • SGLang + DFlash2 block-16 drafter • FP8 KV

Killy

50,467 Aufrufe • vor 2 Monaten

Day-0 Qwen3.8-27B vs Qwen3.6-27B: the voxel pagoda test. Same prompt, one attempt each, single DGX Spark, both NVFP4. 3.8: 75,291 tok · 52m 56s · 23.7 tok/s (vLLM, native MTP) 3.6: 12,117 tok · 5m 13s · 38.8 tok/s (llama.cpp, DFlash K10) 3.8 thought for 160K characters before writing a line of code.
0:15

Day-0 Qwen3.8-27B vs Qwen3.6-27B: the voxel pagoda test. Same prompt, one attempt each, single DGX Spark, both NVFP4. 3.8: 75,291 tok · 52m 56s · 23.7 tok/s (vLLM, native MTP) 3.6: 12,117 tok · 5m 13s · 38.8 tok/s (llama.cpp, DFlash K10) 3.8 thought for 160K characters before writing a line of code.

Wësche

102,648 Aufrufe • vor 2 Monaten

AGI at home Running DeepSeek R1 across my 7 M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)
0:14

AGI at home Running DeepSeek R1 across my 7 M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)

Alex Cheema

1,936,101 Aufrufe • vor 1 Jahr

🐰: i've noticed that when taehyun and soobin are on the same team, the overall quality is higher 🦊: no that’s not it 🦊: when taehyun’s there, the quality is higher 🦊: when you’re mixed in, that’s when it drops a bit 🐰: ◠ ̫◠
0:13

🐰: i've noticed that when taehyun and soobin are on the same team, the overall quality is higher 🦊: no that’s not it 🦊: when taehyun’s there, the quality is higher 🦊: when you’re mixed in, that’s when it drops a bit 🐰: ◠ ̫◠

hourly yeonbin

15,003 Aufrufe • vor 7 Monaten

Got Aleph Alpha's new Kolibri 1 running on a single DGX Spark the day it came out. Their container is x86 only, so I built vLLM 0.29.0 (arm64) + their aleph-alpha-inference plugin. FP8 weights (74 GB) and an FP8 KV cache, 164K context. The FP8 kernels worked on GB10 even though torch doesn't list sm_121. Speed: 48 tok/s solo, 157 tok/s with 8 streams, 5.2K tok/s prompt read. Then I gave it the pagoda prompt 👇 Kolibri 1, 5 tries, thinking on: 3 blank pages from THREE.OrbitControls used the old way 1 blank page, Three.js never loaded 1 runs: floating cubes 👇 Qwen3.6 27B: first try, 12K tokens. Qwen3.8 27B: first try, 75K.
0:15

Got Aleph Alpha's new Kolibri 1 running on a single DGX Spark the day it came out. Their container is x86 only, so I built vLLM 0.29.0 (arm64) + their aleph-alpha-inference plugin. FP8 weights (74 GB) and an FP8 KV cache, 164K context. The FP8 kernels worked on GB10 even though torch doesn't list sm_121. Speed: 48 tok/s solo, 157 tok/s with 8 streams, 5.2K tok/s prompt read. Then I gave it the pagoda prompt 👇 Kolibri 1, 5 tries, thinking on: 3 blank pages from THREE.OrbitControls used the old way 1 blank page, Three.js never loaded 1 runs: floating cubes 👇 Qwen3.6 27B: first try, 12K tokens. Qwen3.8 27B: first try, 75K.

Wësche

14,468 Aufrufe • vor 1 Monat

📹 Qwen image edit fp8 + Wan 2.2 fp8 + lightx2v 8steps lora 🔥 🗺workflow:
0:17

📹 Qwen image edit fp8 + Wan 2.2 fp8 + lightx2v 8steps lora 🔥 🗺workflow:

camenduru

60,982 Aufrufe • vor 1 Jahr

Gemini 3.0 Flash vs Gemini 3.0 Pro Same prompt. Flash is much faster and quality is surprisingly close. Which one do you prefer?
0:17

Gemini 3.0 Flash vs Gemini 3.0 Pro Same prompt. Flash is much faster and quality is surprisingly close. Which one do you prefer?

Mattia

226,692 Aufrufe • vor 10 Monaten

What the hell does Quant actually do? A model is trained in BF16, released in Q8, and we use Q4 because we're told 'its nearly lossless'. Fine, but I was curious about the 'nearly' part, what is actually being lost between BF16 all the way down to Q2? As always, its complicated. Needed a way to visualize the loss, so I had a Qwen 3.8 27B BF16 create a paragraph then had every quant below it do the same and marked the mutations. When a model writes a word, it assigns 'how sure' percentage to each. For example, 'The weather tomorrow is *windy*, a model is 90% sure all the words are correct except the windy part, since it can also be sunny, cold, hot, cloudy, etc. Any words a BF16 model was 90% sure of, every quant down to Q2 rarely changes it. On the other hand, any word that it was only 40~50% sure of, such as windy in our example, are the types of data that gets impacted by the quant process. Okay so what, some words changed, how does this impact anything? I set out to find out, and its all below if you are just as weird as I am about these things.
0:16

What the hell does Quant actually do? A model is trained in BF16, released in Q8, and we use Q4 because we're told 'its nearly lossless'. Fine, but I was curious about the 'nearly' part, what is actually being lost between BF16 all the way down to Q2? As always, its complicated. Needed a way to visualize the loss, so I had a Qwen 3.8 27B BF16 create a paragraph then had every quant below it do the same and marked the mutations. When a model writes a word, it assigns 'how sure' percentage to each. For example, 'The weather tomorrow is *windy*, a model is 90% sure all the words are correct except the windy part, since it can also be sunny, cold, hot, cloudy, etc. Any words a BF16 model was 90% sure of, every quant down to Q2 rarely changes it. On the other hand, any word that it was only 40~50% sure of, such as windy in our example, are the types of data that gets impacted by the quant process. Okay so what, some words changed, how does this impact anything? I set out to find out, and its all below if you are just as weird as I am about these things.

Killy

19,592 Aufrufe • vor 1 Monat

Qwen 3.8 27b nvfp4 vs Grok 4.6 Someone tell me how this local model that is 100x smaller and 10x slower made a better animation?! Elon Musk help us solve this, both with their highest thinking mode (xhigh vs max)
0:15

Qwen 3.8 27b nvfp4 vs Grok 4.6 Someone tell me how this local model that is 100x smaller and 10x slower made a better animation?! Elon Musk help us solve this, both with their highest thinking mode (xhigh vs max)

Wësche

166,698 Aufrufe • vor 2 Monaten

Now you can choose your size and still get the same Pro experience 🙌 The new #Pixel9 Pro and Pro XL have more in common than ever: 👬 Same Super Actua displays 👬 Same processors 👬 Same cameras #MadeByGoogle
0:11

Now you can choose your size and still get the same Pro experience 🙌 The new #Pixel9 Pro and Pro XL have more in common than ever: 👬 Same Super Actua displays 👬 Same processors 👬 Same cameras #MadeByGoogle

Made by Google

24,888 Aufrufe • vor 2 Jahren

Achieving 3,000 tokens/s with 16-parallel inference of Qwen3.6 35B-A3B NVFP4 on a single RTX PRO 6000 WorkStation Edition. I am using ParallelHue—a tool I am developing—to color-code the tokens generated simultaneously via speculative decoding.
0:14

Achieving 3,000 tokens/s with 16-parallel inference of Qwen3.6 35B-A3B NVFP4 on a single RTX PRO 6000 WorkStation Edition. I am using ParallelHue—a tool I am developing—to color-code the tokens generated simultaneously via speculative decoding.

Hikari∣LocalLLM⚡

27,607 Aufrufe • vor 3 Monaten

I tested every Unsloth quant of Qwen3.8 27B on canvas coding, and Q8 was NOT the winner 👀 Setup: 2 prompts: > growing tree animation > paper slicing) 3 runs each, across BF16 (Original) / UD-Q2_K_XL / UD-Q3_K_XL / UD-Q4_K_XL / UD-Q6_K_XL / UD-Q8_K_XL. 🌳 Growing tree test (successes out of 3): > BF16 — 3/3 > Q2 — 1/3 > Q3 — 3/3 > Q4 — 3/3 > Q6 — 3/3 > Q8 — 2/3 📄 Paper slice test (successes out of 3): > BF16 — 1/3 > Q2 — 0/3 > Q3 — 2/3 > Q4 — 2/3 > Q6 — 2/3 > Q8 — 1/3 Takeaway: "bigger quant = better" didn't hold here, Q8 underperformed Q4 and Q6 on both tests. If you're running this model locally, Q6/Q4 looks like the sweet spot: near-full quality at a fraction of the VRAM.
0:15

I tested every Unsloth quant of Qwen3.8 27B on canvas coding, and Q8 was NOT the winner 👀 Setup: 2 prompts: > growing tree animation > paper slicing) 3 runs each, across BF16 (Original) / UD-Q2_K_XL / UD-Q3_K_XL / UD-Q4_K_XL / UD-Q6_K_XL / UD-Q8_K_XL. 🌳 Growing tree test (successes out of 3): > BF16 — 3/3 > Q2 — 1/3 > Q3 — 3/3 > Q4 — 3/3 > Q6 — 3/3 > Q8 — 2/3 📄 Paper slice test (successes out of 3): > BF16 — 1/3 > Q2 — 0/3 > Q3 — 2/3 > Q4 — 2/3 > Q6 — 2/3 > Q8 — 1/3 Takeaway: "bigger quant = better" didn't hold here, Q8 underperformed Q4 and Q6 on both tests. If you're running this model locally, Q6/Q4 looks like the sweet spot: near-full quality at a fraction of the VRAM.

stevibe

82,127 Aufrufe • vor 2 Monaten

i just tested to put GLM-5.2 on my rig. 753B parameter MoE. 2x RTX PRO 6000 Blackwells, Threadripper PRO 9995WX with 1TB DDR5. prefills at 64 tok/s. decode holds at 13-15. system RAM bandwidth is the bottleneck. running UD-Q4_K_XL 4-bit. 69.7% on Aider Polyglot, within two points of BF16. left the 5090 off. mismatched VRAM unbalances the split. this is what local AI looks like now. massive frontier models on a "desktop rig". how awesome is this? and this is the worst it'll ever be.
0:18

i just tested to put GLM-5.2 on my rig. 753B parameter MoE. 2x RTX PRO 6000 Blackwells, Threadripper PRO 9995WX with 1TB DDR5. prefills at 64 tok/s. decode holds at 13-15. system RAM bandwidth is the bottleneck. running UD-Q4_K_XL 4-bit. 69.7% on Aider Polyglot, within two points of BF16. left the 5090 off. mismatched VRAM unbalances the split. this is what local AI looks like now. massive frontier models on a "desktop rig". how awesome is this? and this is the worst it'll ever be.

Samuel Cardillo

48,841 Aufrufe • vor 4 Monaten

You don’t have to use one model (or one provider!) for everything. With Workshop, you can combine frontier and local models in the same workflow. For example: Opus can be the main agent, and delegate specific tasks to Gemma 4 via subagents. Better quality where it matters. Better privacy, speed, and cost where it counts. One workflow, best model for each task.
0:18

You don’t have to use one model (or one provider!) for everything. With Workshop, you can combine frontier and local models in the same workflow. For example: Opus can be the main agent, and delegate specific tasks to Gemma 4 via subagents. Better quality where it matters. Better privacy, speed, and cost where it counts. One workflow, best model for each task.

Workshop AI

1,046,048 Aufrufe • vor 6 Monaten

breaking news: atlas for iphone have been achieved internally. same engine, same quality, same speed. it’s fascinating how much bigger the world feels once you step outside the swiftui bubble.
0:18

breaking news: atlas for iphone have been achieved internally. same engine, same quality, same speed. it’s fascinating how much bigger the world feels once you step outside the swiftui bubble.

alexey

79,534 Aufrufe • vor 4 Monaten

Maguire: "Overall benefit spending as a share of the national economy is about where it was 10yrs ago. The fact [Robertson] has gone for benefits shows it as a highly political attack" And not only is it about the same as 10yrs ago, we spend less than most European countries
0:15

Maguire: "Overall benefit spending as a share of the national economy is about where it was 10yrs ago. The fact [Robertson] has gone for benefits shows it as a highly political attack" And not only is it about the same as 10yrs ago, we spend less than most European countries

Saul Staniforth

122,758 Aufrufe • vor 6 Monaten

📌📌📌if you want the same quality hair at a cheap cheap price ,call this number direct from their supplier and oh Im not affiliated with them 😌073) 450-6281
0:13

📌📌📌if you want the same quality hair at a cheap cheap price ,call this number direct from their supplier and oh Im not affiliated with them 😌073) 450-6281

Reggy

39,028 Aufrufe • vor 4 Monaten

At any point on a curve, there's one circle that fits the curve better than any other. It touches the curve at that point, leans in the same direction, and bends by the same amount. It tells you how sharply the curve is bending at that exact spot, and which way it's leaning. If the curve is bending sharply, the circle is small. If the curve is nearly straight, the circle is huge. The osculating circle of the curve.
0:17

At any point on a curve, there's one circle that fits the curve better than any other. It touches the curve at that point, leans in the same direction, and bends by the same amount. It tells you how sharply the curve is bending at that exact spot, and which way it's leaning. If the curve is bending sharply, the circle is small. If the curve is nearly straight, the circle is huge. The osculating circle of the curve.

The Math Flow

51,317 Aufrufe • vor 2 Monaten

Google’s Gemma 4 E2B running on-device on iPhone 17 Pro Gemma 4 is built from the same research as Gemini 3, has image understanding capabilities and can reason if needed Running at ~40tk/s with MLX optimized for Apple Silicon
0:12

Google’s Gemma 4 E2B running on-device on iPhone 17 Pro Gemma 4 is built from the same research as Gemini 3, has image understanding capabilities and can reason if needed Running at ~40tk/s with MLX optimized for Apple Silicon

Adrien Grondin

1,046,062 Aufrufe • vor 7 Monaten