Qwen3.6-27B dense quality ladder. Same tasks. Same grader. Same... day. How much does compression cost? FP8 - 83.3 · 29GB NVFP4 - 83.0 · ~15GB BF16 - 81.9 · 56GB GPTQ-Pro - 79.4 · 13GB IQ2_XXS - 79.0 · 9.4GB Who wins where: 🥇 Raw quality → FP8 (83.3) 🥇 Practical default → NVFP4 (83.0 at ~15GB) 🥈 Full precision → BF16 is NOT better (81.9) ❌ "Just use any 4-bit" → GPTQ-Pro loses 3.6 pts vs NVFP4 🛟 Emergency tiny → IQ2 still holds at 79.0 Overall winner: If you care about the absolute number: FP8 If you care about running it on a real box: NVFP4 is the overall pick. Same quality class as full precision. Half the size of FP8. ~4× smaller than BF16.show more

454 tok/s on Qwen3.8 27B on single RTX 6000... with +6000 mem overclock Mia’s Qwen3.8-27B NVFP4 W4A4 • SGLang + DFlash2 block-16 drafter • FP8 KVshow more

Day-0 Qwen3.8-27B vs Qwen3.6-27B: the voxel pagoda test. Same... prompt, one attempt each, single DGX Spark, both NVFP4. 3.8: 75,291 tok · 52m 56s · 23.7 tok/s (vLLM, native MTP) 3.6: 12,117 tok · 5m 13s · 38.8 tok/s (llama.cpp, DFlash K10) 3.8 thought for 160K characters before writing a line of code.show more

AGI at home Running DeepSeek R1 across my 7... M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)show more

🐰: i've noticed that when taehyun and soobin are... on the same team, the overall quality is higher 🦊: no that’s not it 🦊: when taehyun’s there, the quality is higher 🦊: when you’re mixed in, that’s when it drops a bit 🐰: ◠ ̫◠show more

Got Aleph Alpha's new Kolibri 1 running on a... single DGX Spark the day it came out. Their container is x86 only, so I built vLLM 0.29.0 (arm64) + their aleph-alpha-inference plugin. FP8 weights (74 GB) and an FP8 KV cache, 164K context. The FP8 kernels worked on GB10 even though torch doesn't list sm_121. Speed: 48 tok/s solo, 157 tok/s with 8 streams, 5.2K tok/s prompt read. Then I gave it the pagoda prompt 👇 Kolibri 1, 5 tries, thinking on: 3 blank pages from THREE.OrbitControls used the old way 1 blank page, Three.js never loaded 1 runs: floating cubes 👇 Qwen3.6 27B: first try, 12K tokens. Qwen3.8 27B: first try, 75K.show more

📹 Qwen image edit fp8 + Wan 2.2 fp8... + lightx2v 8steps lora 🔥 🗺workflow:show more

Gemini 3.0 Flash vs Gemini 3.0 Pro Same prompt.... Flash is much faster and quality is surprisingly close. Which one do you prefer?show more

Qwen 3.8 27b nvfp4 vs Grok 4.6 Someone tell... me how this local model that is 100x smaller and 10x slower made a better animation?! Elon Musk help us solve this, both with their highest thinking mode (xhigh vs max)show more

What the hell does Quant actually do? A model... is trained in BF16, released in Q8, and we use Q4 because we're told 'its nearly lossless'. Fine, but I was curious about the 'nearly' part, what is actually being lost between BF16 all the way down to Q2? As always, its complicated. Needed a way to visualize the loss, so I had a Qwen 3.8 27B BF16 create a paragraph then had every quant below it do the same and marked the mutations. When a model writes a word, it assigns 'how sure' percentage to each. For example, 'The weather tomorrow is *windy*, a model is 90% sure all the words are correct except the windy part, since it can also be sunny, cold, hot, cloudy, etc. Any words a BF16 model was 90% sure of, every quant down to Q2 rarely changes it. On the other hand, any word that it was only 40~50% sure of, such as windy in our example, are the types of data that gets impacted by the quant process. Okay so what, some words changed, how does this impact anything? I set out to find out, and its all below if you are just as weird as I am about these things.show more

Now you can choose your size and still get... the same Pro experience 🙌 The new #Pixel9 Pro and Pro XL have more in common than ever: 👬 Same Super Actua displays 👬 Same processors 👬 Same cameras #MadeByGoogleshow more

Achieving 3,000 tokens/s with 16-parallel inference of Qwen3.6 35B-A3B... NVFP4 on a single RTX PRO 6000 WorkStation Edition. I am using ParallelHue—a tool I am developing—to color-code the tokens generated simultaneously via speculative decoding.show more

I tested every Unsloth quant of Qwen3.8 27B on... canvas coding, and Q8 was NOT the winner 👀 Setup: 2 prompts: > growing tree animation > paper slicing) 3 runs each, across BF16 (Original) / UD-Q2_K_XL / UD-Q3_K_XL / UD-Q4_K_XL / UD-Q6_K_XL / UD-Q8_K_XL. 🌳 Growing tree test (successes out of 3): > BF16 — 3/3 > Q2 — 1/3 > Q3 — 3/3 > Q4 — 3/3 > Q6 — 3/3 > Q8 — 2/3 📄 Paper slice test (successes out of 3): > BF16 — 1/3 > Q2 — 0/3 > Q3 — 2/3 > Q4 — 2/3 > Q6 — 2/3 > Q8 — 1/3 Takeaway: "bigger quant = better" didn't hold here, Q8 underperformed Q4 and Q6 on both tests. If you're running this model locally, Q6/Q4 looks like the sweet spot: near-full quality at a fraction of the VRAM.show more

You don’t have to use one model (or one... provider!) for everything. With Workshop, you can combine frontier and local models in the same workflow. For example: Opus can be the main agent, and delegate specific tasks to Gemma 4 via subagents. Better quality where it matters. Better privacy, speed, and cost where it counts. One workflow, best model for each task.show more

i just tested to put GLM-5.2 on my rig.... 753B parameter MoE. 2x RTX PRO 6000 Blackwells, Threadripper PRO 9995WX with 1TB DDR5. prefills at 64 tok/s. decode holds at 13-15. system RAM bandwidth is the bottleneck. running UD-Q4_K_XL 4-bit. 69.7% on Aider Polyglot, within two points of BF16. left the 5090 off. mismatched VRAM unbalances the split. this is what local AI looks like now. massive frontier models on a "desktop rig". how awesome is this? and this is the worst it'll ever be.show more

Maguire: "Overall benefit spending as a share of the... national economy is about where it was 10yrs ago. The fact [Robertson] has gone for benefits shows it as a highly political attack" And not only is it about the same as 10yrs ago, we spend less than most European countriesshow more

breaking news: atlas for iphone have been achieved internally.... same engine, same quality, same speed. it’s fascinating how much bigger the world feels once you step outside the swiftui bubble.show more

📌📌📌if you want the same quality hair at a... cheap cheap price ,call this number direct from their supplier and oh Im not affiliated with them 😌073) 450-6281show more

At any point on a curve, there's one circle... that fits the curve better than any other. It touches the curve at that point, leans in the same direction, and bends by the same amount. It tells you how sharply the curve is bending at that exact spot, and which way it's leaning. If the curve is bending sharply, the circle is small. If the curve is nearly straight, the circle is huge. The osculating circle of the curve.show more

Google’s Gemma 4 E2B running on-device on iPhone 17... Pro Gemma 4 is built from the same research as Gemini 3, has image understanding capabilities and can reason if needed Running at ~40tk/s with MLX optimized for Apple Siliconshow more