Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Qwen 3.8 Flash Next Gyro Coder 🚢 - Optimized for (agentic) coding workflows - Fits 24 GB VRAM - Python KLD 0.165 - 21 GB total size - 50% of experts kept

Laurent Zuijdwijk

1,561 subscribers

13,922 views • 4 days ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

19 Comments

Kyle Hessling4 days ago

Speed is looking awesome on a 5090! Great work! Looking forward to doing some more tests!

Marko4 days ago

21 GB of a 24 GB card is gone before the conversation starts, so the agent loop's real limit is the little that is left.

nardoblanc3 days ago

fitv in 24gb is just wow bro Congrats

HelloHello3 days ago

gyro M is my everyday used, I have dual 3060 + P100 (gen 2x1) with 24 gb system Ram, 30-40 token per sec. satisfied for me with the model intelligence.

Laurent Zuijdwijk3 days ago

That is a great result! Would love to hear your opinion about gyro coder.

Mark Murphy3 days ago

Agentic coding tune that fits in 24GB is a nice one. I'm on Qwen 3.8 27B at home, so this is going on my list to try.

JuggyBob4 days ago

@grok is this better than the GSQ RCO quant that strata uses?

🤖Sergio3 days ago

How does it compare with Qwen 3.8 27B Q4?

El Minero Sudaka4 days ago

Make a 4 bit please

Laurent Zuijdwijk3 days ago

Why? Just run the ful Gyro M model.

君子中庸3 days ago

WHAT? 50% of experts kept it means...?

TolkienWindow4 days ago

look pretty cool, how does it do with spatial reasoning with all those experts trimmed?

Laurent Zuijdwijk3 days ago

Look at the examples in the video. I paid extra care to get the spatial aspect right and keep world knowledge intact.

TolkienWindow3 days ago

good deal, thank you for taking the time to get that tuned! <3

Latent Node3 days ago

Curious about the calibration set: if experts were pruned by routing frequency on mostly Python, TypeScript and Rust prompts may lean on the dropped ones. What does the KLD look like there?

Laurent Zuijdwijk3 days ago

Experts weren't just dropped by routing frequency. I spent a lot of time keeping world and spatial knowledge intact while keeping the coding KLD as low as possible. KLD was measured at every step across Python, web, C++, Rust and shell. Routing frequency was just the starting point.

Latent Node3 days ago

Thanks for the detail. Measuring KLD across five languages at every step is more careful than most pruned releases. A per-language KLD table on the model card would help people pick the right build for their stack.

Laurent Zuijdwijk3 days ago

I am adding this to the model card as well.

Alfred3 days ago

Great work, as always. How does it compare to Gyro-M? Did it have to sacrifice a lot of world knowledge?

Related Videos

The VRAM barrier is officially dead. I just ran Qwen 3.8 Flash Next (MoE) 125B A6B with a 250,000 context window on a single 24GB RTX 4090. 21 tokens/sec decode. 364 t/s prefill. no mtp. no dflash. no kv cache quantization! We are running datacenter models on consumer hardware. Tested on Ubuntu 22 | CUDA 13.0 | PCIe 4.0 x16 | 110 GB DDR4 System RAM with a continuous 28k prompt across all runs. ### The Benchmarks & Scaling # 1. Hybrid Offload (-ncmoe 40 @ 80k Context) Offloaded 40 expert layers to the GPU, pushing VRAM to the ceiling. ./build/bin/llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf -c 80000 --port 8080 -v --fit off -b 4096 -ub 4096 -ncmoe 40 Prefill: 383.85 t/s | Decode: 22.52 t/s Footprint: 23.85 GB VRAM | 97 GB RAM # 2. Full CPU MoE Offload (-cmoe @ 80k Context) Pinned all 512 expert layers to DDR4 RAM (-cmoe), keeping attention on the 4090. llama.cpp flags: (Same as above, replace -ncmoe 40 with -cmoe) Prefill: 355.72 t/s | Decode: 20.84 t/s Footprint: 11.66 GB VRAM (12GB+ VRAM freed up!) | 110 GB RAM # 3. The 180,000 Context Run Prefill: 357.75 t/s | Decode: 20.98 t/s | VRAM: 15.6 GB | RAM: 110 GB # 4. The 250,000 Context Absolute Ceiling ./build/bin/llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf -c 250000 --port 8080 -v --fit off -b 4096 -ub 4096 -cmoe Prefill: 364.29 t/s | Decode: 20.97 t/s Footprint: 18.3 GB VRAM (Still ~5.7 GB of VRAM headroom!) | 110 GB RAM ### Key Insights: -b 4096 -ub 4096: doubles the prompt ingestion from ~150 to 364+ t/s. -cmoe Free Lunch: Shifting expert layers to DDR4 RAM slashes VRAM from 24GB to 11.6GB with virtually zero decode penalty (22.5 -> 20.9 t/s), enabling the 250k context ceiling. Qwen 3.8 Flash-Next (UD-Q4_K_XL) is a massive 111.4 GB model split across 4 shards. To run this architecture, you must build from the experimental PR branch (#27742) by Daniel Han: git clone && cd llama.cpp git fetch origin pull/27742/head:qwen-next && git checkout qwen-next cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native -DBUILD_SHARED_LIBS=OFF cmake --build build --config Release -j $(nproc) --target llama-server A single 4090 paired with 100 GB of cheap DDR4 RAM will comfortably serve production grade 125B inference. While Qwen 3.8 27B (dense) still holds the crown for single 3090/4090 rigs, Flash Next proves 125B hybrid models are officially viable on consumer hardware. Hugging Face GGUF link and complete performance telemetry graphs are dropped in the replies below. GLM 5.3 Flash VS Qwen 3.8 Flash Next, which one takes the open weights crown this week?

The VRAM barrier is officially dead. I just ran Qwen 3.8 Flash Next (MoE) 125B A6B with a 250,000 context window on a single 24GB RTX 4090. 21 tokens/sec decode. 364 t/s prefill. no mtp. no dflash. no kv cache quantization! We are running datacenter models on consumer hardware. Tested on Ubuntu 22 | CUDA 13.0 | PCIe 4.0 x16 | 110 GB DDR4 System RAM with a continuous 28k prompt across all runs. ### The Benchmarks & Scaling # 1. Hybrid Offload (-ncmoe 40 @ 80k Context) Offloaded 40 expert layers to the GPU, pushing VRAM to the ceiling. ./build/bin/llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf -c 80000 --port 8080 -v --fit off -b 4096 -ub 4096 -ncmoe 40 Prefill: 383.85 t/s | Decode: 22.52 t/s Footprint: 23.85 GB VRAM | 97 GB RAM # 2. Full CPU MoE Offload (-cmoe @ 80k Context) Pinned all 512 expert layers to DDR4 RAM (-cmoe), keeping attention on the 4090. llama.cpp flags: (Same as above, replace -ncmoe 40 with -cmoe) Prefill: 355.72 t/s | Decode: 20.84 t/s Footprint: 11.66 GB VRAM (12GB+ VRAM freed up!) | 110 GB RAM # 3. The 180,000 Context Run Prefill: 357.75 t/s | Decode: 20.98 t/s | VRAM: 15.6 GB | RAM: 110 GB # 4. The 250,000 Context Absolute Ceiling ./build/bin/llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf -c 250000 --port 8080 -v --fit off -b 4096 -ub 4096 -cmoe Prefill: 364.29 t/s | Decode: 20.97 t/s Footprint: 18.3 GB VRAM (Still ~5.7 GB of VRAM headroom!) | 110 GB RAM ### Key Insights: -b 4096 -ub 4096: doubles the prompt ingestion from ~150 to 364+ t/s. -cmoe Free Lunch: Shifting expert layers to DDR4 RAM slashes VRAM from 24GB to 11.6GB with virtually zero decode penalty (22.5 -> 20.9 t/s), enabling the 250k context ceiling. Qwen 3.8 Flash-Next (UD-Q4_K_XL) is a massive 111.4 GB model split across 4 shards. To run this architecture, you must build from the experimental PR branch (#27742) by Daniel Han: git clone && cd llama.cpp git fetch origin pull/27742/head:qwen-next && git checkout qwen-next cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native -DBUILD_SHARED_LIBS=OFF cmake --build build --config Release -j $(nproc) --target llama-server A single 4090 paired with 100 GB of cheap DDR4 RAM will comfortably serve production grade 125B inference. While Qwen 3.8 27B (dense) still holds the crown for single 3090/4090 rigs, Flash Next proves 125B hybrid models are officially viable on consumer hardware. Hugging Face GGUF link and complete performance telemetry graphs are dropped in the replies below. GLM 5.3 Flash VS Qwen 3.8 Flash Next, which one takes the open weights crown this week?

Alok

1,059,537 views • 2 months ago

Qwen 3.8 27B is already agentic lore. So we ran two Qwen 3.8 27B models in one local build. We stacked the Qwen models using the new Windows Hermes Agent desktop bot mode running on our 128 GB Ryzen AI Max+ processor based agent computer. See how Ryzen AI Max+ architecture keeps local Qwen agent builders, building.

Qwen 3.8 27B is already agentic lore. So we ran two Qwen 3.8 27B models in one local build. We stacked the Qwen models using the new Windows Hermes Agent desktop bot mode running on our 128 GB Ryzen AI Max+ processor based agent computer. See how Ryzen AI Max+ architecture keeps local Qwen agent builders, building.

AMD Ryzen

45,176 views • 1 month ago

Founder & CEO of Flutterwave GB (OluGB 🦋), in our one-size-fits-all Dreamers Fila.

Founder & CEO of Flutterwave GB (OluGB 🦋), in our one-size-fits-all Dreamers Fila.

S.O™️

42,205 views • 17 days ago

Which flash model wins? DeepSeek v4.1 flash, Qwen 3.8 flash, or GLM 5.3 flash 🤯

Which flash model wins? DeepSeek v4.1 flash, Qwen 3.8 flash, or GLM 5.3 flash 🤯

GMI Cloud

15,052 views • 1 month ago

Qwen 3.8 vs GLM 5.3 vs Gemini 3.8 Flash vs DeepSeek v4.1 Flash Prompt: Build an Angry Birds clone Qwen 3.8 took $0.49 in 3 min GLM 5.3 took $0.46 in 2 min Gemini 3.8 Flash took $0.29 in 2 min 23 sec DeepSeek v4.1 Flash took $0.11 in 1 min 28 sec Which one wins for you?

Qwen 3.8 vs GLM 5.3 vs Gemini 3.8 Flash vs DeepSeek v4.1 Flash Prompt: Build an Angry Birds clone Qwen 3.8 took $0.49 in 3 min GLM 5.3 took $0.46 in 2 min Gemini 3.8 Flash took $0.29 in 2 min 23 sec DeepSeek v4.1 Flash took $0.11 in 1 min 28 sec Which one wins for you?

Bhavy☄️

27,799 views • 1 month ago

Videos of Hermes Desktop using 2 x DGX Spark with: - GLM 5.3 Flash EXL3 - Qwen 3.8 Flash Next - DeepSeek V4 0731 Flash All powered by Mia recipes and her sparkDash!

Videos of Hermes Desktop using 2 x DGX Spark with: - GLM 5.3 Flash EXL3 - Qwen 3.8 Flash Next - DeepSeek V4 0731 Flash All powered by Mia recipes and her sparkDash!

Ivan Fioravanti ᯅ

45,257 views • 1 month ago

Introducing MTPLX V2.12: The fastest way to run local AI now even faster and with more models for lower memory macs and the new M5 Ultra. Upgrades: - 100 TPS Qwen Flash Next sustained on long coding (M5 max). - 2x Prefill speeds Qwen Flash Next: 872 --> 1695 TPS. - Requests with images now use the compiled kernels for max speeds. New Models: - Qwen Flash Next MTPLX Optimized Quality: 8 bit model built for the M5 Ultra and M3 Ultra. - Bonsai 2 27B MTPLX Optimized Speed: Works on 16gb and 24gb macs with compiled kernels and MTP running as fast as possible. 64 TPS. - MiMo V2.6 Qwen 3.5 9B MTPLX Optimized Speed: My personal favourite. Mimos magic distilled into a 9B model easily runnable at high speeds on a 16gb mac. 107 TPS. +40 bug fixes and improvements in real world coding.

Introducing MTPLX V2.12: The fastest way to run local AI now even faster and with more models for lower memory macs and the new M5 Ultra. Upgrades: - 100 TPS Qwen Flash Next sustained on long coding (M5 max). - 2x Prefill speeds Qwen Flash Next: 872 --> 1695 TPS. - Requests with images now use the compiled kernels for max speeds. New Models: - Qwen Flash Next MTPLX Optimized Quality: 8 bit model built for the M5 Ultra and M3 Ultra. - Bonsai 2 27B MTPLX Optimized Speed: Works on 16gb and 24gb macs with compiled kernels and MTP running as fast as possible. 64 TPS. - MiMo V2.6 Qwen 3.5 9B MTPLX Optimized Speed: My personal favourite. Mimos magic distilled into a 9B model easily runnable at high speeds on a 16gb mac. 107 TPS. +40 bug fixes and improvements in real world coding.

Youssof Al Toukhi

21,486 views • 1 month ago

I still can't believe I have this quality of AI on one DGX Spark. NVIDIA AI Qwen 3.8 Flash Next NVFP4, on vLLM

I still can't believe I have this quality of AI on one DGX Spark. NVIDIA AI Qwen 3.8 Flash Next NVFP4, on vLLM

Base Camp Bernie

16,314 views • 1 month ago

Project Maya v1.1.16 🐳 - DeepSeek-V4.1-Flash-1017 (748B) on your own PC - Maya-S, our own quant: 178 GB, 92.6% same next token as the original - NVIDIA + AMD, Linux + Windows - Experts from RAM/SSD: runs below its size

Project Maya v1.1.16 🐳 - DeepSeek-V4.1-Flash-1017 (748B) on your own PC - Maya-S, our own quant: 178 GB, 92.6% same next token as the original - NVIDIA + AMD, Linux + Windows - Experts from RAM/SSD: runs below its size

Peasant Smith

44,826 views • 6 days ago

Qwen 3.8 Max is 50% off on Venice for a limited time, in collaboration with Qwen.

Qwen 3.8 Max is 50% off on Venice for a limited time, in collaboration with Qwen.

Venice

141,647 views • 2 months ago

Alibaba just open-sourced Qwen 3.6-35B A3B 35B MoE model (only ~3B active), multimodal, agentic coding, Apache 2.0. This isn’t just efficient—it’s built for real workflows. Open models are catching up fast.

Alibaba just open-sourced Qwen 3.6-35B A3B 35B MoE model (only ~3B active), multimodal, agentic coding, Apache 2.0. This isn’t just efficient—it’s built for real workflows. Open models are catching up fast.

Mervin Praison

13,063 views • 6 months ago

Blockchain can enhance your existing Python projects. Here’s why blockchain is important: #python #blockchain #AlgoKit #Algorand #smartcontracts #defi #cryptocurrency #programming #developer #programmer #softwaredeveloper #coding #coder #computerscience #dapp #nft #Algos #perawallet #pera #programming #developer #programmer #coding #coder #softwaredeveloper #computerscience #project

Blockchain can enhance your existing Python projects. Here’s why blockchain is important: #python #blockchain #AlgoKit #Algorand #smartcontracts #defi #cryptocurrency #programming #developer #programmer #softwaredeveloper #coding #coder #computerscience #dapp #nft #Algos #perawallet #pera #programming #developer #programmer #coding #coder #softwaredeveloper #computerscience #project

Python Developer

11,991 views • 2 years ago

MTPLX V2.10 is out! I put a lot of work into this one. Highlights: - Qwen 3.8 Next Support Added: Bare Speed (4 bit) and Optimised Speed (4 bit/8 bit) models added running at 50 - 80 TPS sustained at high contexts. (expect speed improvements in the coming days) - Qwen 3.8 Next mmap: The 32 GB n-gram table streams from SSD via mmap instead of RAM. Both models fit a 96 GB Mac with 0 speed impact. - Qwen 3.8 27B Decode & Prefill boost EVERYWHERE: +15% TPS at 3k context, +29% at 88k, +54% at 147k. Prefill +41%. - 48 GB Macs went from 3 to 4 TPS in swap @ 30k context to 33 TPS. - KV Cache has been solved!: 8 bit and 4 bit cache now barely impacts speed instead of a 50% decline. - Coding agents wall time optimisations: file task dropped from 150s to 44s, and mid-session first token went from ~2s to 0.11s. - CLI UX improvements + 34 bug fixes. My thoughts on Qwen Next? Incredible model. Enormously better at design, and creative tasks with scary good vision. It created what I think is the best flappy bird. maybe even better than the Fable version. It had various difficulties and skins, moving pillars, and upgrades. It also had the most memeified look. Incredibly impressed. However, it is slightly worse than 27B at having code just work straight away. But it's personality is also far better than 27B. Here is the Flappy bird it made below! Made with Qwen Next Optimized Speed one shot@ 57 TPS:

MTPLX V2.10 is out! I put a lot of work into this one. Highlights: - Qwen 3.8 Next Support Added: Bare Speed (4 bit) and Optimised Speed (4 bit/8 bit) models added running at 50 - 80 TPS sustained at high contexts. (expect speed improvements in the coming days) - Qwen 3.8 Next mmap: The 32 GB n-gram table streams from SSD via mmap instead of RAM. Both models fit a 96 GB Mac with 0 speed impact. - Qwen 3.8 27B Decode & Prefill boost EVERYWHERE: +15% TPS at 3k context, +29% at 88k, +54% at 147k. Prefill +41%. - 48 GB Macs went from 3 to 4 TPS in swap @ 30k context to 33 TPS. - KV Cache has been solved!: 8 bit and 4 bit cache now barely impacts speed instead of a 50% decline. - Coding agents wall time optimisations: file task dropped from 150s to 44s, and mid-session first token went from ~2s to 0.11s. - CLI UX improvements + 34 bug fixes. My thoughts on Qwen Next? Incredible model. Enormously better at design, and creative tasks with scary good vision. It created what I think is the best flappy bird. maybe even better than the Fable version. It had various difficulties and skins, moving pillars, and upgrades. It also had the most memeified look. Incredibly impressed. However, it is slightly worse than 27B at having code just work straight away. But it's personality is also far better than 27B. Here is the Flappy bird it made below! Made with Qwen Next Optimized Speed one shot@ 57 TPS:

Youssof Al Toukhi

21,228 views • 1 month ago

Qwen just quietly dropped the most dangerous open model of 2026 Qwen3.6-Plus. native 1M context. agentic workflows. multimodal. half the size of competitors. and it's free on OpenRouter right now. you're not ready for this breakdown 🧵

Qwen just quietly dropped the most dangerous open model of 2026 Qwen3.6-Plus. native 1M context. agentic workflows. multimodal. half the size of competitors. and it's free on OpenRouter right now. you're not ready for this breakdown 🧵

Farhan

53,820 views • 6 months ago

20 hours later and I’m done! Qwen 3.8 27B @ 73 TPS (peak) on a Macbook pro M5 max. MTPLX V2.7 out now with 3 new models. - Bare Speed: short burst decode, good for chat. - Optimized Speed: higher quality and faster on long coding. - Optimized Quality: 30% slower but super!

20 hours later and I’m done! Qwen 3.8 27B @ 73 TPS (peak) on a Macbook pro M5 max. MTPLX V2.7 out now with 3 new models. - Bare Speed: short burst decode, good for chat. - Optimized Speed: higher quality and faster on long coding. - Optimized Quality: 30% slower but super!

Youssof Al Toukhi

97,435 views • 2 months ago

768 GB de VRAM en un chasis 4U refrigerado por líquida Ocho tarjetas RTX PRO 6000 Blackwell, de 96 GB cada una Comino Grando

768 GB de VRAM en un chasis 4U refrigerado por líquida Ocho tarjetas RTX PRO 6000 Blackwell, de 96 GB cada una Comino Grando

elhacker.NET

14,272 views • 4 months ago

After months of work, today I’m releasing Foundation-1. A SOTA text-to-sample model built specifically for music production workflows. It may also be the most advanced AI sample generator currently available - open or closed. • ~7 GB VRAM • Entirely local • 100% free 😁

After months of work, today I’m releasing Foundation-1. A SOTA text-to-sample model built specifically for music production workflows. It may also be the most advanced AI sample generator currently available - open or closed. • ~7 GB VRAM • Entirely local • 100% free 😁

JD | RoyalCities

119,279 views • 7 months ago

There’s no one-size-fits-all SKU for agentic AI. “The customer environments are diverse, the workloads are diverse, the constraints are diverse.” AMD’s Madhu Rangarajan explains why end-to-end agentic workflows will continue to require different compute configurations:

There’s no one-size-fits-all SKU for agentic AI. “The customer environments are diverse, the workloads are diverse, the constraints are diverse.” AMD’s Madhu Rangarajan explains why end-to-end agentic workflows will continue to require different compute configurations:

AMD

28,585 views • 2 months ago

Running 1-bit Qwen 3.8 Flash Next (79GB) on a MacBook Pro M5 Max 64GB at 30 tok/s 🤯 It ran a 8-minute agent loop with 6 web searches, 3 Python runs and 5 sourced tables. Run local models via Atomic Chat! GGUF:

Running 1-bit Qwen 3.8 Flash Next (79GB) on a MacBook Pro M5 Max 64GB at 30 tok/s 🤯 It ran a 8-minute agent loop with 6 web searches, 3 Python runs and 5 sourced tables. Run local models via Atomic Chat! GGUF:

atomic.chat

104,339 views • 1 month ago