正在加载视频...

视频加载失败

返回首页

MLX Swift LLM example works with: - Mistral / Llama - Phi-2 - Qwen 1.5 - Starcoder 2 Quick-start: Qwen 1.5 0.5B runs pretty fast in 16-bit on my iPhone 14, no quantization needed:

awnihannun's profile picture

Awni Hannun

44,814 subscribers

30,441 次观看 • 2 年前 •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

DeepSeek R1 distilled to Qwen 1.5B easily runs on my iPhone 16 with MLX swift. Here's the 4-bit model reasoning entirely on device at almost 60 toks/sec:
0:19

DeepSeek R1 distilled to Qwen 1.5B easily runs on my iPhone 16 with MLX swift. Here's the 4-bit model reasoning entirely on device at almost 60 toks/sec:

Awni Hannun

1,139,656 次观看 • 1 年前

Qwen 3.5 0.8B, Gated DeltaNet attention is running on Apple Neural Engine ~56 t/s in LUT6 quantization with some room for optimization left. It is CoreML, Swift and IOSurface on M4Pro. It will slow down as we increase context, but not by much. I think Private API opens the way to integrate ANE with GPU/MLX and possibly some MoE.
0:19

Qwen 3.5 0.8B, Gated DeltaNet attention is running on Apple Neural Engine ~56 t/s in LUT6 quantization with some room for optimization left. It is CoreML, Swift and IOSurface on M4Pro. It will slow down as we increase context, but not by much. I think Private API opens the way to integrate ANE with GPU/MLX and possibly some MoE.

Anemll

13,589 次观看 • 4 个月前

Another demo of the iPhone 17 Pro’s on-device LLM performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quants
0:11

Another demo of the iPhone 17 Pro’s on-device LLM performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quants

Adrien Grondin

46,205 次观看 • 9 个月前

DeepSeek-R1-0528-Qwen3-8B running on iPhone 16 with MLX
0:16

DeepSeek-R1-0528-Qwen3-8B running on iPhone 16 with MLX

ℏεsam

17,285 次观看 • 1 年前

Qwen 3 family is shipping with the upcoming 0.4.0 version of React Native ExecuTorch. 🎉 The 1.7B variant reaches 30 tokens per second on iPhone 16 Pro!
0:18

Qwen 3 family is shipping with the upcoming 0.4.0 version of React Native ExecuTorch. 🎉 The 1.7B variant reaches 30 tokens per second on iPhone 16 Pro!

Software Mansion

11,637 次观看 • 1 年前

Inference on UOMI is now free. Start building with frontier open-source models at no cost: • MiniMax M2.7 • Qwen 3.6 27B • Qwen 3.6 35B A3B • Google Gemma 4 26B A4B • Google Gemma 4 31B Powered by the UOMI Inference Network. Start inferencing:
0:14

Inference on UOMI is now free. Start building with frontier open-source models at no cost: • MiniMax M2.7 • Qwen 3.6 27B • Qwen 3.6 35B A3B • Google Gemma 4 26B A4B • Google Gemma 4 31B Powered by the UOMI Inference Network. Start inferencing:

Uomi

12,479 次观看 • 1 个月前

Managed to get Ling Mini 16B (1.4B active) running on my iPhone Air. It runs very fast with MLX. It's a DWQ of Ling Mini quantized to 3 bits-per-weight. A 16B model running on an Air at this speed is pretty awesome:
0:16

Managed to get Ling Mini 16B (1.4B active) running on my iPhone Air. It runs very fast with MLX. It's a DWQ of Ling Mini quantized to 3 bits-per-weight. A 16B model running on an Air at this speed is pretty awesome:

Awni Hannun

92,422 次观看 • 9 个月前

DeepSeek R1 Qwen 7B 4bit M2 Ultra vs M4 Max on Apple MLX 🤫 Let them think... (video 4x in center part) M2 Ultra: 114.9 tokens per sec M4 Max (14"): 88.3 tokens per sec
0:16

DeepSeek R1 Qwen 7B 4bit M2 Ultra vs M4 Max on Apple MLX 🤫 Let them think... (video 4x in center part) M2 Ultra: 114.9 tokens per sec M4 Max (14"): 88.3 tokens per sec

Ivan Fioravanti ᯅ

60,200 次观看 • 1 年前

MLX news: MOSS-TTS-Local Transformer 1.5 available on mlx-audio now! Thanks to Prince Canuma and Lucas Newman Audio on! 🔉
0:11

MLX news: MOSS-TTS-Local Transformer 1.5 available on mlx-audio now! Thanks to Prince Canuma and Lucas Newman Audio on! 🔉

Ivan Fioravanti ᯅ

20,292 次观看 • 17 天前

Here Qwen 3 0.6B again on Italian wines 82.5%! The best commercial LLM I tested in the past was scoring 75.6% (Sonnet 3.5)
0:19

Here Qwen 3 0.6B again on Italian wines 82.5%! The best commercial LLM I tested in the past was scoring 75.6% (Sonnet 3.5)

Ivan Fioravanti ᯅ

25,367 次观看 • 1 年前

BOOM 🚨: Tencent’s Hunyuan Turbo S just landed in the Top 8 globally on Chatbot Arena and is now #2 in China, just behind DeepSeek. Even Google ex-CEO Eric Schmidt says China had no foundation models two years ago — now it has three: DeepSeek, Qwen, Hunyuan — on par with OpenAI’s O.1. At #GoogleIO, all three scored high on the global leaderboard like #DeepSeek, #Qwen, #Hunyuan Tencent Hy isn’t just racing — it’s rewriting the leaderboard. #HunyuanTurboS #TencentAI #LLM #AIRevolution
0:18

BOOM 🚨: Tencent’s Hunyuan Turbo S just landed in the Top 8 globally on Chatbot Arena and is now #2 in China, just behind DeepSeek. Even Google ex-CEO Eric Schmidt says China had no foundation models two years ago — now it has three: DeepSeek, Qwen, Hunyuan — on par with OpenAI’s O.1. At #GoogleIO, all three scored high on the global leaderboard like #DeepSeek, #Qwen, #Hunyuan Tencent Hy isn’t just racing — it’s rewriting the leaderboard. #HunyuanTurboS #TencentAI #LLM #AIRevolution

Emily Watson | AI Tools & Tech News

24,814 次观看 • 1 年前

Qwen3.7-Max on AI/ML API - built for the agent era GPQA Diamond (92.4), HMMT (97.1), Apex (44.5) Sustains 35+ hours of autonomous execution Works with Claude Code, Qwen Code & more Comment Qwen to get Free promo code
0:19

Qwen3.7-Max on AI/ML API - built for the agent era GPQA Diamond (92.4), HMMT (97.1), Apex (44.5) Sustains 35+ hours of autonomous execution Works with Claude Code, Qwen Code & more Comment Qwen to get Free promo code

AI/ML API

35,288 次观看 • 1 个月前

pip install spectralquant ✂️ Up to 6.62x KV cache compression for LLMs and transformers. Same model. Faster outputs. Smaller KV cache. Try now (2 mins): - KV cache integration via Hugging Face's DynamicCache - Three presets: 5.95x (paper), 6.55x (validated), 6.68x (edge) - Mistral 7B / Qwen 2.5 7B / Llama 3.1 8B verified - Pure PyTorch + future CUDA kernel support - Auto-calibration from a bundled corpus 📰 Paper: 💻 Code, quickstart, and benchmarks: #LLM #Inference #PyTorch #OpenSource #MachineLearning #LLM #KVCache #Inference
0:15

pip install spectralquant ✂️ Up to 6.62x KV cache compression for LLMs and transformers. Same model. Faster outputs. Smaller KV cache. Try now (2 mins): - KV cache integration via Hugging Face's DynamicCache - Three presets: 5.95x (paper), 6.55x (validated), 6.68x (edge) - Mistral 7B / Qwen 2.5 7B / Llama 3.1 8B verified - Pure PyTorch + future CUDA kernel support - Auto-calibration from a bundled corpus 📰 Paper: 💻 Code, quickstart, and benchmarks: #LLM #Inference #PyTorch #OpenSource #MachineLearning #LLM #KVCache #Inference

ani

16,583 次观看 • 1 个月前

Someone got DeepSeek-R1-0528-Qwen3-8B running on an iPhone 16 Pro using MLX. It runs but takes ages to respond, and the phone gets hot fast. 8B models on phones aren't sci-fi anymore. via u/adrgrondin on r/LocalLLaMA
0:16

Someone got DeepSeek-R1-0528-Qwen3-8B running on an iPhone 16 Pro using MLX. It runs but takes ages to respond, and the phone gets hot fast. 8B models on phones aren't sci-fi anymore. via u/adrgrondin on r/LocalLLaMA

👋 Jan

82,742 次观看 • 1 年前

WIP Thinking UI for Qwen 3 thinking mode, running locally on iPhone. Tried to have something similar to Grok web thinking UI. Made 100% with SwiftUI.
0:12

WIP Thinking UI for Qwen 3 thinking mode, running locally on iPhone. Tried to have something similar to Grok web thinking UI. Made 100% with SwiftUI.

Adrien Grondin

64,271 次观看 • 1 年前