MLX Swift LLM example works with: - Mistral /... Llama - Phi-2 - Qwen 1.5 - Starcoder 2 Quick-start: Qwen 1.5 0.5B runs pretty fast in 16-bit on my iPhone 14, no quantization needed:show more

DeepSeek R1 distilled to Qwen 1.5B easily runs on... my iPhone 16 with MLX swift. Here's the 4-bit model reasoning entirely on device at almost 60 toks/sec:show more

Qwen 3.5 0.8B, Gated DeltaNet attention is running on... Apple Neural Engine ~56 t/s in LUT6 quantization with some room for optimization left. It is CoreML, Swift and IOSurface on M4Pro. It will slow down as we increase context, but not by much. I think Private API opens the way to integrate ANE with GPU/MLX and possibly some MoE.show more

Another demo of the iPhone 17 Pro’s on-device LLM... performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quantsshow more

DeepSeek-R1-0528-Qwen3-8B running on iPhone 16 with MLX

Qwen 3 family is shipping with the upcoming 0.4.0... version of React Native ExecuTorch. 🎉 The 1.7B variant reaches 30 tokens per second on iPhone 16 Pro!show more

Inference on UOMI is now free. Start building with... frontier open-source models at no cost: • MiniMax M2.7 • Qwen 3.6 27B • Qwen 3.6 35B A3B • Google Gemma 4 26B A4B • Google Gemma 4 31B Powered by the UOMI Inference Network. Start inferencing:show more

Managed to get Ling Mini 16B (1.4B active) running... on my iPhone Air. It runs very fast with MLX. It's a DWQ of Ling Mini quantized to 3 bits-per-weight. A 16B model running on an Air at this speed is pretty awesome:show more

DeepSeek R1 Qwen 7B 4bit M2 Ultra vs M4... Max on Apple MLX 🤫 Let them think... (video 4x in center part) M2 Ultra: 114.9 tokens per sec M4 Max (14"): 88.3 tokens per secshow more

MLX news: MOSS-TTS-Local Transformer 1.5 available on mlx-audio now!... Thanks to Prince Canuma and Lucas Newman Audio on! 🔉show more

Here Qwen 3 0.6B again on Italian wines 82.5%!... The best commercial LLM I tested in the past was scoring 75.6% (Sonnet 3.5)show more

BOOM 🚨: Tencent’s Hunyuan Turbo S just landed in... the Top 8 globally on Chatbot Arena and is now #2 in China, just behind DeepSeek. Even Google ex-CEO Eric Schmidt says China had no foundation models two years ago — now it has three: DeepSeek, Qwen, Hunyuan — on par with OpenAI’s O.1. At #GoogleIO, all three scored high on the global leaderboard like #DeepSeek, #Qwen, #Hunyuan Tencent Hy isn’t just racing — it’s rewriting the leaderboard. #HunyuanTurboS #TencentAI #LLM #AIRevolutionshow more

Qwen3.7-Max on AI/ML API - built for the agent... era GPQA Diamond (92.4), HMMT (97.1), Apex (44.5) Sustains 35+ hours of autonomous execution Works with Claude Code, Qwen Code & more Comment Qwen to get Free promo codeshow more

pip install spectralquant ✂️ Up to 6.62x KV cache... compression for LLMs and transformers. Same model. Faster outputs. Smaller KV cache. Try now (2 mins): - KV cache integration via Hugging Face's DynamicCache - Three presets: 5.95x (paper), 6.55x (validated), 6.68x (edge) - Mistral 7B / Qwen 2.5 7B / Llama 3.1 8B verified - Pure PyTorch + future CUDA kernel support - Auto-calibration from a bundled corpus 📰 Paper: 💻 Code, quickstart, and benchmarks: #LLM #Inference #PyTorch #OpenSource #MachineLearning #LLM #KVCache #Inferenceshow more

Someone got DeepSeek-R1-0528-Qwen3-8B running on an iPhone 16 Pro... using MLX. It runs but takes ages to respond, and the phone gets hot fast. 8B models on phones aren't sci-fi anymore. via u/adrgrondin on r/LocalLLaMAshow more

WIP Thinking UI for Qwen 3 thinking mode, running... locally on iPhone. Tried to have something similar to Grok web thinking UI. Made 100% with SwiftUI.show more