Zhijian Liu
@zhijianliu_ • 9,743 subscribers
Building something new (@inco_ai). Faculty @UCSanDiego (running https://t.co/dxXyoN9zSi). Prev: PhD @MIT. RS @NVIDIA. Efficient AI. Views are my own.
Shorts
Videos
Reasoning LLMs generate very long chains-of-thought, so even small quantization errors add up. With AWQ, Qwen3-4B drops 71.0 → 68.2 on MMLU-Pro (~4% relative loss). 😬 ParoQuant fixes this! It keeps only the critical rotation pairs and fuses everything into a single kernel. Recovers most of the lost reasoning accuracy with minimal overhead — so 4-bit models stay strong at reasoning. 💪💪
Zhijian Liu171,597 görüntüleme • 8 ay önce
ParoQuant just got a big upgrade 🚀 ✅ Supports the new Qwen3.5 models ⚡ Now runs on MLX (fast local inference on Apple Silicon) 🧠 Preserves reasoning quality with 4-bit quantization We also built an agent demo running locally on my 4-year-old M2 Max. Can't wait to upgrade to an M5 Max and see what kind of magic we can do. ✨
Zhijian Liu49,826 görüntüleme • 7 ay önce
⚡ Speed of flash. Just 2 days after launch, DFlash is already running in SGLang (SGLang). With serving-engine support, we can now unlock speedup with higher concurrency, and we’ve quickly worked on a new demo based on it. We'll be cooking up more and better draft models over the next few weeks.🔥 Stay tuned!
Zhijian Liu21,233 görüntüleme • 9 ay önce
Daha fazla içerik yok.