Shuo Yang
@Andy_ShuoYang • 5,028 subscribers
3rd year phd at Berkeley; Efficient ML System;
Videos
Your gaming PC can now serve frontier models at interactive speed using official checkpoints without extreme quantization! Qwen3.6 35B → 8GB RTX 4060 laptop @ 39 tok/s DeepSeek-V4-Flash 284B → RTX 5090 desktop @ 22-25 tok/s GLM-5.2 753B → RTX PRO 6000 workstation @ 15 tok/s Run your claude code or codex now with frontier model for $0 Meet FreeToken 🧵
Shuo Yang1,100,928 просмотров • 1 месяц назад
Qwen3.8-Flash-Next now runs on a single RTX 5090 at 68.3 tok/s — no extreme quantization, no speculative decoding. · Production-level checkpoint: GB300-validated NVFP4 checkpoint from RadixArk · 63GB host RAM — less than what 1-bit quants of this model need · The 51GB n-gram table? It lives on your NVMe, at ~0.5% throughput cost In the video: one prompt → a playable minecraft-style world, then FreeToken on my 5090 desktop fixing a real FreeToken bug in 10.5 minutes. Meet FreeToken 🧵
Shuo Yang64,307 просмотров • 1 месяц назад
Больше нет контента для загрузки