Wësche's banner

Wësche

@WescheNex1q • 1,588 subscribers

Day time artist and night time AI enthusiast. Building & benchmarking frontier LLMs on 4x DGX Spark clusters + Mac. Creator of Vesica Studio. Houston

Shorts

Qwen3.6-27B dense quality ladder. Same tasks. Same grader. Same day. How much does compression cost? FP8 - 83.3 · 29GB NVFP4 - 83.0 · ~15GB BF16 - 81.9 · 56GB GPTQ-Pro - 79.4 · 13GB IQ2_XXS - 79.0 · 9.4GB Who wins where: 🥇 Raw quality → FP8 (83.3) 🥇 Practical default → NVFP4 (83.0 at ~15GB) 🥈 Full precision → BF16 is NOT better (81.9) ❌ "Just use any 4-bit" → GPTQ-Pro loses 3.6 pts vs NVFP4 🛟 Emergency tiny → IQ2 still holds at 79.0 Overall winner: If you care about the absolute number: FP8 If you care about running it on a real box: NVFP4 is the overall pick. Same quality class as full precision. Half the size of FP8. ~4× smaller than BF16.

Qwen3.6-27B dense quality ladder. Same tasks. Same grader. Same day. How much does compression cost? FP8 - 83.3 · 29GB NVFP4 - 83.0 · ~15GB BF16 - 81.9 · 56GB GPTQ-Pro - 79.4 · 13GB IQ2_XXS - 79.0 · 9.4GB Who wins where: 🥇 Raw quality → FP8 (83.3) 🥇 Practical default → NVFP4 (83.0 at ~15GB) 🥈 Full precision → BF16 is NOT better (81.9) ❌ "Just use any 4-bit" → GPTQ-Pro loses 3.6 pts vs NVFP4 🛟 Emergency tiny → IQ2 still holds at 79.0 Overall winner: If you care about the absolute number: FP8 If you care about running it on a real box: NVFP4 is the overall pick. Same quality class as full precision. Half the size of FP8. ~4× smaller than BF16.

24,599 просмотров

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

One DGX Spark. Qwen3.6-35B 64 users. 700+ tok/s. 32,768 tokens in 54 seconds. 38W Each user has their own prompt and their own KV cache, and vLLM batches every active stream through the GPU each step. Recipe → NVIDIA AI

One DGX Spark. Qwen3.6-35B 64 users. 700+ tok/s. 32,768 tokens in 54 seconds. 38W Each user has their own prompt and their own KV cache, and vLLM batches every active stream through the GPU each step. Recipe → NVIDIA AI

163,881 просмотров • 21 дней назад

16 people chatting with Qwen3.6-35B at once ONE DGX Spark. This is a real capture, not a mockup: every token you see replays at its actual measured arrival time. Peaks ~440 tok/s total. A single user gets 105 tok/s. NVFP4 + MTP-3 on vLLM.

16 people chatting with Qwen3.6-35B at once ONE DGX Spark. This is a real capture, not a mockup: every token you see replays at its actual measured arrival time. Peaks ~440 tok/s total. A single user gets 105 tok/s. NVFP4 + MTP-3 on vLLM.

26,094 просмотров • 22 дней назад

Больше нет контента для загрузки