Загрузка видео...

Не удалось загрузить видео

На главную

Qwen 3.8 Flash Next Gyro Coder 🚢 - Optimized for (agentic) coding workflows - Fits 24 GB VRAM - Python KLD 0.165 - 21 GB total size - 50% of experts kept

laurent_zw's profile picture

Laurent Zuijdwijk

1,561 subscribers

13,922 просмотров • 4 дней назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 19

Фото профиля Kyle Hessling
Kyle Hessling4 дней назад

Speed is looking awesome on a 5090! Great work! Looking forward to doing some more tests!

Фото профиля Marko
Marko4 дней назад

21 GB of a 24 GB card is gone before the conversation starts, so the agent loop's real limit is the little that is left.

Фото профиля nardoblanc
nardoblanc3 дней назад

fitv in 24gb is just wow bro Congrats

Фото профиля HelloHello
HelloHello3 дней назад

gyro M is my everyday used, I have dual 3060 + P100 (gen 2x1) with 24 gb system Ram, 30-40 token per sec. satisfied for me with the model intelligence.

Фото профиля Laurent Zuijdwijk
Laurent Zuijdwijk3 дней назад

That is a great result! Would love to hear your opinion about gyro coder.

Фото профиля Mark Murphy
Mark Murphy3 дней назад

Agentic coding tune that fits in 24GB is a nice one. I'm on Qwen 3.8 27B at home, so this is going on my list to try.

Фото профиля JuggyBob
JuggyBob4 дней назад

@grok is this better than the GSQ RCO quant that strata uses?

Фото профиля 🤖Sergio
🤖Sergio3 дней назад

How does it compare with Qwen 3.8 27B Q4?

Фото профиля El Minero Sudaka
El Minero Sudaka4 дней назад

Make a 4 bit please

Фото профиля Laurent Zuijdwijk
Laurent Zuijdwijk3 дней назад

Why? Just run the ful Gyro M model.

Фото профиля 君子中庸
君子中庸3 дней назад

WHAT? 50% of experts kept it means...?

Фото профиля TolkienWindow
TolkienWindow4 дней назад

look pretty cool, how does it do with spatial reasoning with all those experts trimmed?

Фото профиля Laurent Zuijdwijk
Laurent Zuijdwijk3 дней назад

Look at the examples in the video. I paid extra care to get the spatial aspect right and keep world knowledge intact.

Фото профиля TolkienWindow
TolkienWindow3 дней назад

good deal, thank you for taking the time to get that tuned! <3

Фото профиля Latent Node
Latent Node3 дней назад

Curious about the calibration set: if experts were pruned by routing frequency on mostly Python, TypeScript and Rust prompts may lean on the dropped ones. What does the KLD look like there?

Фото профиля Laurent Zuijdwijk
Laurent Zuijdwijk3 дней назад

Experts weren't just dropped by routing frequency. I spent a lot of time keeping world and spatial knowledge intact while keeping the coding KLD as low as possible. KLD was measured at every step across Python, web, C++, Rust and shell. Routing frequency was just the starting point.

Фото профиля Latent Node
Latent Node3 дней назад

Thanks for the detail. Measuring KLD across five languages at every step is more careful than most pruned releases. A per-language KLD table on the model card would help people pick the right build for their stack.

Фото профиля Laurent Zuijdwijk
Laurent Zuijdwijk3 дней назад

I am adding this to the model card as well.

Фото профиля Alfred
Alfred3 дней назад

Great work, as always. How does it compare to Gyro-M? Did it have to sacrifice a lot of world knowledge?

Похожие видео

The VRAM barrier is officially dead. I just ran Qwen 3.8 Flash Next (MoE) 125B A6B with a 250,000 context window on a single 24GB RTX 4090. 21 tokens/sec decode. 364 t/s prefill. no mtp. no dflash. no kv cache quantization! We are running datacenter models on consumer hardware. Tested on Ubuntu 22 | CUDA 13.0 | PCIe 4.0 x16 | 110 GB DDR4 System RAM with a continuous 28k prompt across all runs. ### The Benchmarks & Scaling # 1. Hybrid Offload (-ncmoe 40 @ 80k Context) Offloaded 40 expert layers to the GPU, pushing VRAM to the ceiling. ./build/bin/llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf -c 80000 --port 8080 -v --fit off -b 4096 -ub 4096 -ncmoe 40 Prefill: 383.85 t/s | Decode: 22.52 t/s Footprint: 23.85 GB VRAM | 97 GB RAM # 2. Full CPU MoE Offload (-cmoe @ 80k Context) Pinned all 512 expert layers to DDR4 RAM (-cmoe), keeping attention on the 4090. llama.cpp flags: (Same as above, replace -ncmoe 40 with -cmoe) Prefill: 355.72 t/s | Decode: 20.84 t/s Footprint: 11.66 GB VRAM (12GB+ VRAM freed up!) | 110 GB RAM # 3. The 180,000 Context Run Prefill: 357.75 t/s | Decode: 20.98 t/s | VRAM: 15.6 GB | RAM: 110 GB # 4. The 250,000 Context Absolute Ceiling ./build/bin/llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf -c 250000 --port 8080 -v --fit off -b 4096 -ub 4096 -cmoe Prefill: 364.29 t/s | Decode: 20.97 t/s Footprint: 18.3 GB VRAM (Still ~5.7 GB of VRAM headroom!) | 110 GB RAM ### Key Insights: -b 4096 -ub 4096: doubles the prompt ingestion from ~150 to 364+ t/s. -cmoe Free Lunch: Shifting expert layers to DDR4 RAM slashes VRAM from 24GB to 11.6GB with virtually zero decode penalty (22.5 -> 20.9 t/s), enabling the 250k context ceiling. Qwen 3.8 Flash-Next (UD-Q4_K_XL) is a massive 111.4 GB model split across 4 shards. To run this architecture, you must build from the experimental PR branch (#27742) by Daniel Han: git clone && cd llama.cpp git fetch origin pull/27742/head:qwen-next && git checkout qwen-next cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native -DBUILD_SHARED_LIBS=OFF cmake --build build --config Release -j $(nproc) --target llama-server A single 4090 paired with 100 GB of cheap DDR4 RAM will comfortably serve production grade 125B inference. While Qwen 3.8 27B (dense) still holds the crown for single 3090/4090 rigs, Flash Next proves 125B hybrid models are officially viable on consumer hardware. Hugging Face GGUF link and complete performance telemetry graphs are dropped in the replies below. GLM 5.3 Flash VS Qwen 3.8 Flash Next, which one takes the open weights crown this week?
0:20

The VRAM barrier is officially dead. I just ran Qwen 3.8 Flash Next (MoE) 125B A6B with a 250,000 context window on a single 24GB RTX 4090. 21 tokens/sec decode. 364 t/s prefill. no mtp. no dflash. no kv cache quantization! We are running datacenter models on consumer hardware. Tested on Ubuntu 22 | CUDA 13.0 | PCIe 4.0 x16 | 110 GB DDR4 System RAM with a continuous 28k prompt across all runs. ### The Benchmarks & Scaling # 1. Hybrid Offload (-ncmoe 40 @ 80k Context) Offloaded 40 expert layers to the GPU, pushing VRAM to the ceiling. ./build/bin/llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf -c 80000 --port 8080 -v --fit off -b 4096 -ub 4096 -ncmoe 40 Prefill: 383.85 t/s | Decode: 22.52 t/s Footprint: 23.85 GB VRAM | 97 GB RAM # 2. Full CPU MoE Offload (-cmoe @ 80k Context) Pinned all 512 expert layers to DDR4 RAM (-cmoe), keeping attention on the 4090. llama.cpp flags: (Same as above, replace -ncmoe 40 with -cmoe) Prefill: 355.72 t/s | Decode: 20.84 t/s Footprint: 11.66 GB VRAM (12GB+ VRAM freed up!) | 110 GB RAM # 3. The 180,000 Context Run Prefill: 357.75 t/s | Decode: 20.98 t/s | VRAM: 15.6 GB | RAM: 110 GB # 4. The 250,000 Context Absolute Ceiling ./build/bin/llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf -c 250000 --port 8080 -v --fit off -b 4096 -ub 4096 -cmoe Prefill: 364.29 t/s | Decode: 20.97 t/s Footprint: 18.3 GB VRAM (Still ~5.7 GB of VRAM headroom!) | 110 GB RAM ### Key Insights: -b 4096 -ub 4096: doubles the prompt ingestion from ~150 to 364+ t/s. -cmoe Free Lunch: Shifting expert layers to DDR4 RAM slashes VRAM from 24GB to 11.6GB with virtually zero decode penalty (22.5 -> 20.9 t/s), enabling the 250k context ceiling. Qwen 3.8 Flash-Next (UD-Q4_K_XL) is a massive 111.4 GB model split across 4 shards. To run this architecture, you must build from the experimental PR branch (#27742) by Daniel Han: git clone && cd llama.cpp git fetch origin pull/27742/head:qwen-next && git checkout qwen-next cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native -DBUILD_SHARED_LIBS=OFF cmake --build build --config Release -j $(nproc) --target llama-server A single 4090 paired with 100 GB of cheap DDR4 RAM will comfortably serve production grade 125B inference. While Qwen 3.8 27B (dense) still holds the crown for single 3090/4090 rigs, Flash Next proves 125B hybrid models are officially viable on consumer hardware. Hugging Face GGUF link and complete performance telemetry graphs are dropped in the replies below. GLM 5.3 Flash VS Qwen 3.8 Flash Next, which one takes the open weights crown this week?

Alok

1,059,537 просмотров • 2 месяцев назад

Qwen 3.8 27B is already agentic lore. So we ran two Qwen 3.8 27B models in one local build. We stacked the Qwen models using the new Windows Hermes Agent desktop bot mode running on our 128 GB Ryzen AI Max+ processor based agent computer. See how Ryzen AI Max+ architecture keeps local Qwen agent builders, building.
0:52

Qwen 3.8 27B is already agentic lore. So we ran two Qwen 3.8 27B models in one local build. We stacked the Qwen models using the new Windows Hermes Agent desktop bot mode running on our 128 GB Ryzen AI Max+ processor based agent computer. See how Ryzen AI Max+ architecture keeps local Qwen agent builders, building.

AMD Ryzen

45,176 просмотров • 1 месяц назад

Founder & CEO of Flutterwave GB (OluGB 🦋), in our one-size-fits-all Dreamers Fila.
0:26

Founder & CEO of Flutterwave GB (OluGB 🦋), in our one-size-fits-all Dreamers Fila.

S.O™️

42,205 просмотров • 17 дней назад

Which flash model wins? DeepSeek v4.1 flash, Qwen 3.8 flash, or GLM 5.3 flash 🤯
0:41

Which flash model wins? DeepSeek v4.1 flash, Qwen 3.8 flash, or GLM 5.3 flash 🤯

GMI Cloud

15,052 просмотров • 1 месяц назад

Qwen 3.8 vs GLM 5.3 vs Gemini 3.8 Flash vs DeepSeek v4.1 Flash Prompt: Build an Angry Birds clone Qwen 3.8 took $0.49 in 3 min GLM 5.3 took $0.46 in 2 min Gemini 3.8 Flash took $0.29 in 2 min 23 sec DeepSeek v4.1 Flash took $0.11 in 1 min 28 sec Which one wins for you?
0:24

Qwen 3.8 vs GLM 5.3 vs Gemini 3.8 Flash vs DeepSeek v4.1 Flash Prompt: Build an Angry Birds clone Qwen 3.8 took $0.49 in 3 min GLM 5.3 took $0.46 in 2 min Gemini 3.8 Flash took $0.29 in 2 min 23 sec DeepSeek v4.1 Flash took $0.11 in 1 min 28 sec Which one wins for you?

Bhavy☄️

27,799 просмотров • 1 месяц назад

Videos of Hermes Desktop using 2 x DGX Spark with: - GLM 5.3 Flash EXL3 - Qwen 3.8 Flash Next - DeepSeek V4 0731 Flash All powered by Mia recipes and her sparkDash!
0:56

Videos of Hermes Desktop using 2 x DGX Spark with: - GLM 5.3 Flash EXL3 - Qwen 3.8 Flash Next - DeepSeek V4 0731 Flash All powered by Mia recipes and her sparkDash!

Ivan Fioravanti ᯅ

45,257 просмотров • 1 месяц назад

Introducing MTPLX V2.12: The fastest way to run local AI now even faster and with more models for lower memory macs and the new M5 Ultra. Upgrades: - 100 TPS Qwen Flash Next sustained on long coding (M5 max). - 2x Prefill speeds Qwen Flash Next: 872 --> 1695 TPS. - Requests with images now use the compiled kernels for max speeds. New Models: - Qwen Flash Next MTPLX Optimized Quality: 8 bit model built for the M5 Ultra and M3 Ultra. - Bonsai 2 27B MTPLX Optimized Speed: Works on 16gb and 24gb macs with compiled kernels and MTP running as fast as possible. 64 TPS. - MiMo V2.6 Qwen 3.5 9B MTPLX Optimized Speed: My personal favourite. Mimos magic distilled into a 9B model easily runnable at high speeds on a 16gb mac. 107 TPS. +40 bug fixes and improvements in real world coding.
0:24

Introducing MTPLX V2.12: The fastest way to run local AI now even faster and with more models for lower memory macs and the new M5 Ultra. Upgrades: - 100 TPS Qwen Flash Next sustained on long coding (M5 max). - 2x Prefill speeds Qwen Flash Next: 872 --> 1695 TPS. - Requests with images now use the compiled kernels for max speeds. New Models: - Qwen Flash Next MTPLX Optimized Quality: 8 bit model built for the M5 Ultra and M3 Ultra. - Bonsai 2 27B MTPLX Optimized Speed: Works on 16gb and 24gb macs with compiled kernels and MTP running as fast as possible. 64 TPS. - MiMo V2.6 Qwen 3.5 9B MTPLX Optimized Speed: My personal favourite. Mimos magic distilled into a 9B model easily runnable at high speeds on a 16gb mac. 107 TPS. +40 bug fixes and improvements in real world coding.

Youssof Al Toukhi

21,486 просмотров • 1 месяц назад

I still can't believe I have this quality of AI on one DGX Spark. NVIDIA AI Qwen 3.8 Flash Next NVFP4, on vLLM
0:50

I still can't believe I have this quality of AI on one DGX Spark. NVIDIA AI Qwen 3.8 Flash Next NVFP4, on vLLM

Base Camp Bernie

16,314 просмотров • 1 месяц назад

Project Maya v1.1.16 🐳 - DeepSeek-V4.1-Flash-1017 (748B) on your own PC - Maya-S, our own quant: 178 GB, 92.6% same next token as the original - NVIDIA + AMD, Linux + Windows - Experts from RAM/SSD: runs below its size
0:27

Project Maya v1.1.16 🐳 - DeepSeek-V4.1-Flash-1017 (748B) on your own PC - Maya-S, our own quant: 178 GB, 92.6% same next token as the original - NVIDIA + AMD, Linux + Windows - Experts from RAM/SSD: runs below its size

Peasant Smith

44,826 просмотров • 6 дней назад

Qwen 3.8 Max is 50% off on Venice for a limited time, in collaboration with Qwen.
0:32

Qwen 3.8 Max is 50% off on Venice for a limited time, in collaboration with Qwen.

Venice

141,647 просмотров • 2 месяцев назад

Alibaba just open-sourced Qwen 3.6-35B A3B 35B MoE model (only ~3B active), multimodal, agentic coding, Apache 2.0. This isn’t just efficient—it’s built for real workflows. Open models are catching up fast.
1:49

Alibaba just open-sourced Qwen 3.6-35B A3B 35B MoE model (only ~3B active), multimodal, agentic coding, Apache 2.0. This isn’t just efficient—it’s built for real workflows. Open models are catching up fast.

Mervin Praison

13,063 просмотров • 6 месяцев назад

Blockchain can enhance your existing Python projects. Here’s why blockchain is important: #python #blockchain #AlgoKit #Algorand #smartcontracts #defi #cryptocurrency #programming #developer #programmer #softwaredeveloper #coding #coder #computerscience #dapp #nft #Algos #perawallet #pera #programming #developer #programmer #coding #coder #softwaredeveloper #computerscience #project
0:50

Blockchain can enhance your existing Python projects. Here’s why blockchain is important: #python #blockchain #AlgoKit #Algorand #smartcontracts #defi #cryptocurrency #programming #developer #programmer #softwaredeveloper #coding #coder #computerscience #dapp #nft #Algos #perawallet #pera #programming #developer #programmer #coding #coder #softwaredeveloper #computerscience #project

Python Developer

11,991 просмотров • 2 лет назад

MTPLX V2.10 is out! I put a lot of work into this one. Highlights: - Qwen 3.8 Next Support Added: Bare Speed (4 bit) and Optimised Speed (4 bit/8 bit) models added running at 50 - 80 TPS sustained at high contexts. (expect speed improvements in the coming days) - Qwen 3.8 Next mmap: The 32 GB n-gram table streams from SSD via mmap instead of RAM. Both models fit a 96 GB Mac with 0 speed impact. - Qwen 3.8 27B Decode & Prefill boost EVERYWHERE: +15% TPS at 3k context, +29% at 88k, +54% at 147k. Prefill +41%. - 48 GB Macs went from 3 to 4 TPS in swap @ 30k context to 33 TPS. - KV Cache has been solved!: 8 bit and 4 bit cache now barely impacts speed instead of a 50% decline. - Coding agents wall time optimisations: file task dropped from 150s to 44s, and mid-session first token went from ~2s to 0.11s. - CLI UX improvements + 34 bug fixes. My thoughts on Qwen Next? Incredible model. Enormously better at design, and creative tasks with scary good vision. It created what I think is the best flappy bird. maybe even better than the Fable version. It had various difficulties and skins, moving pillars, and upgrades. It also had the most memeified look. Incredibly impressed. However, it is slightly worse than 27B at having code just work straight away. But it's personality is also far better than 27B. Here is the Flappy bird it made below! Made with Qwen Next Optimized Speed one shot@ 57 TPS:
1:16

MTPLX V2.10 is out! I put a lot of work into this one. Highlights: - Qwen 3.8 Next Support Added: Bare Speed (4 bit) and Optimised Speed (4 bit/8 bit) models added running at 50 - 80 TPS sustained at high contexts. (expect speed improvements in the coming days) - Qwen 3.8 Next mmap: The 32 GB n-gram table streams from SSD via mmap instead of RAM. Both models fit a 96 GB Mac with 0 speed impact. - Qwen 3.8 27B Decode & Prefill boost EVERYWHERE: +15% TPS at 3k context, +29% at 88k, +54% at 147k. Prefill +41%. - 48 GB Macs went from 3 to 4 TPS in swap @ 30k context to 33 TPS. - KV Cache has been solved!: 8 bit and 4 bit cache now barely impacts speed instead of a 50% decline. - Coding agents wall time optimisations: file task dropped from 150s to 44s, and mid-session first token went from ~2s to 0.11s. - CLI UX improvements + 34 bug fixes. My thoughts on Qwen Next? Incredible model. Enormously better at design, and creative tasks with scary good vision. It created what I think is the best flappy bird. maybe even better than the Fable version. It had various difficulties and skins, moving pillars, and upgrades. It also had the most memeified look. Incredibly impressed. However, it is slightly worse than 27B at having code just work straight away. But it's personality is also far better than 27B. Here is the Flappy bird it made below! Made with Qwen Next Optimized Speed one shot@ 57 TPS:

Youssof Al Toukhi

21,228 просмотров • 1 месяц назад

Qwen just quietly dropped the most dangerous open model of 2026 Qwen3.6-Plus. native 1M context. agentic workflows. multimodal. half the size of competitors. and it's free on OpenRouter right now. you're not ready for this breakdown 🧵
0:35

Qwen just quietly dropped the most dangerous open model of 2026 Qwen3.6-Plus. native 1M context. agentic workflows. multimodal. half the size of competitors. and it's free on OpenRouter right now. you're not ready for this breakdown 🧵

Farhan

53,820 просмотров • 6 месяцев назад

20 hours later and I’m done! Qwen 3.8 27B @ 73 TPS (peak) on a Macbook pro M5 max. MTPLX V2.7 out now with 3 new models. - Bare Speed: short burst decode, good for chat. - Optimized Speed: higher quality and faster on long coding. - Optimized Quality: 30% slower but super!
0:47

20 hours later and I’m done! Qwen 3.8 27B @ 73 TPS (peak) on a Macbook pro M5 max. MTPLX V2.7 out now with 3 new models. - Bare Speed: short burst decode, good for chat. - Optimized Speed: higher quality and faster on long coding. - Optimized Quality: 30% slower but super!

Youssof Al Toukhi

97,435 просмотров • 2 месяцев назад

768 GB de VRAM en un chasis 4U refrigerado por líquida Ocho tarjetas RTX PRO 6000 Blackwell, de 96 GB cada una Comino Grando
1:25

768 GB de VRAM en un chasis 4U refrigerado por líquida Ocho tarjetas RTX PRO 6000 Blackwell, de 96 GB cada una Comino Grando

elhacker.NET

14,272 просмотров • 4 месяцев назад

After months of work, today I’m releasing Foundation-1. A SOTA text-to-sample model built specifically for music production workflows. It may also be the most advanced AI sample generator currently available - open or closed. • ~7 GB VRAM • Entirely local • 100% free 😁
5:31

After months of work, today I’m releasing Foundation-1. A SOTA text-to-sample model built specifically for music production workflows. It may also be the most advanced AI sample generator currently available - open or closed. • ~7 GB VRAM • Entirely local • 100% free 😁

JD | RoyalCities

119,279 просмотров • 7 месяцев назад

There’s no one-size-fits-all SKU for agentic AI. “The customer environments are diverse, the workloads are diverse, the constraints are diverse.” AMD’s Madhu Rangarajan explains why end-to-end agentic workflows will continue to require different compute configurations:
1:54

There’s no one-size-fits-all SKU for agentic AI. “The customer environments are diverse, the workloads are diverse, the constraints are diverse.” AMD’s Madhu Rangarajan explains why end-to-end agentic workflows will continue to require different compute configurations:

AMD

28,585 просмотров • 2 месяцев назад

Running 1-bit Qwen 3.8 Flash Next (79GB) on a MacBook Pro M5 Max 64GB at 30 tok/s 🤯 It ran a 8-minute agent loop with 6 web searches, 3 Python runs and 5 sourced tables. Run local models via Atomic Chat! GGUF:
1:14

Running 1-bit Qwen 3.8 Flash Next (79GB) on a MacBook Pro M5 Max 64GB at 30 tok/s 🤯 It ran a 8-minute agent loop with 6 web searches, 3 Python runs and 5 sourced tables. Run local models via Atomic Chat! GGUF:

atomic.chat

104,339 просмотров • 1 месяц назад