Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

MLX Swift LLM example works with: - Mistral / Llama - Phi-2 - Qwen 1.5 - Starcoder 2 Quick-start: Qwen 1.5 0.5B runs pretty fast in 16-bit on my iPhone 14, no quantization needed:

awnihannun's profile picture

Awni Hannun

44,814 subscribers

30,441 Aufrufe • vor 2 Jahren •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

DeepSeek R1 distilled to Qwen 1.5B easily runs on my iPhone 16 with MLX swift. Here's the 4-bit model reasoning entirely on device at almost 60 toks/sec:
0:19

DeepSeek R1 distilled to Qwen 1.5B easily runs on my iPhone 16 with MLX swift. Here's the 4-bit model reasoning entirely on device at almost 60 toks/sec:

Awni Hannun

1,139,656 Aufrufe • vor 1 Jahr

Helium 2B running locally on an iPhone 16 Pro at 28 tok/s, faster than you can read, all thanks to mlx-swift with q4 quantization!
0:13

Helium 2B running locally on an iPhone 16 Pro at 28 tok/s, faster than you can read, all thanks to mlx-swift with q4 quantization!

kyutai

59,240 Aufrufe • vor 1 Jahr

I was exploring MLX Swift and created a workout assistant to train for my runs! 🏃🏽‍♂️ It uses my workout data from HealthKit for personalization and Qwen 2.5 3B for local inference!
0:13

I was exploring MLX Swift and created a workout assistant to train for my runs! 🏃🏽‍♂️ It uses my workout data from HealthKit for personalization and Qwen 2.5 3B for local inference!

Rudrank Riyam

19,965 Aufrufe • vor 1 Jahr

Qwen 3.5 0.8B, Gated DeltaNet attention is running on Apple Neural Engine ~56 t/s in LUT6 quantization with some room for optimization left. It is CoreML, Swift and IOSurface on M4Pro. It will slow down as we increase context, but not by much. I think Private API opens the way to integrate ANE with GPU/MLX and possibly some MoE.
0:19

Qwen 3.5 0.8B, Gated DeltaNet attention is running on Apple Neural Engine ~56 t/s in LUT6 quantization with some room for optimization left. It is CoreML, Swift and IOSurface on M4Pro. It will slow down as we increase context, but not by much. I think Private API opens the way to integrate ANE with GPU/MLX and possibly some MoE.

Anemll

13,589 Aufrufe • vor 3 Monaten

Another demo of the iPhone 17 Pro’s on-device LLM performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quants
0:11

Another demo of the iPhone 17 Pro’s on-device LLM performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quants

Adrien Grondin

46,205 Aufrufe • vor 9 Monaten

DeepSeek-R1-0528-Qwen3-8B running on iPhone 16 with MLX
0:16

DeepSeek-R1-0528-Qwen3-8B running on iPhone 16 with MLX

ℏεsam

17,285 Aufrufe • vor 1 Jahr

Qwen 3 family is shipping with the upcoming 0.4.0 version of React Native ExecuTorch. 🎉 The 1.7B variant reaches 30 tokens per second on iPhone 16 Pro!
0:18

Qwen 3 family is shipping with the upcoming 0.4.0 version of React Native ExecuTorch. 🎉 The 1.7B variant reaches 30 tokens per second on iPhone 16 Pro!

Software Mansion

11,637 Aufrufe • vor 1 Jahr

Managed to get Ling Mini 16B (1.4B active) running on my iPhone Air. It runs very fast with MLX. It's a DWQ of Ling Mini quantized to 3 bits-per-weight. A 16B model running on an Air at this speed is pretty awesome:
0:16

Managed to get Ling Mini 16B (1.4B active) running on my iPhone Air. It runs very fast with MLX. It's a DWQ of Ling Mini quantized to 3 bits-per-weight. A 16B model running on an Air at this speed is pretty awesome:

Awni Hannun

92,422 Aufrufe • vor 9 Monaten

DeepSeek R1 Qwen 7B 4bit M2 Ultra vs M4 Max on Apple MLX 🤫 Let them think... (video 4x in center part) M2 Ultra: 114.9 tokens per sec M4 Max (14"): 88.3 tokens per sec
0:16

DeepSeek R1 Qwen 7B 4bit M2 Ultra vs M4 Max on Apple MLX 🤫 Let them think... (video 4x in center part) M2 Ultra: 114.9 tokens per sec M4 Max (14"): 88.3 tokens per sec

Ivan Fioravanti ᯅ

60,200 Aufrufe • vor 1 Jahr

Zuckerberg watching you use Qwen instead of LLaMA > via /r/rlocallama
0:11

Zuckerberg watching you use Qwen instead of LLaMA > via /r/rlocallama

Chubby♨️

300,751 Aufrufe • vor 1 Jahr

Here Qwen 3 0.6B again on Italian wines 82.5%! The best commercial LLM I tested in the past was scoring 75.6% (Sonnet 3.5)
0:19

Here Qwen 3 0.6B again on Italian wines 82.5%! The best commercial LLM I tested in the past was scoring 75.6% (Sonnet 3.5)

Ivan Fioravanti ᯅ

25,367 Aufrufe • vor 1 Jahr

MLX news: MOSS-TTS-Local Transformer 1.5 available on mlx-audio now! Thanks to Prince Canuma and Lucas Newman Audio on! 🔉
0:11

MLX news: MOSS-TTS-Local Transformer 1.5 available on mlx-audio now! Thanks to Prince Canuma and Lucas Newman Audio on! 🔉

Ivan Fioravanti ᯅ

20,292 Aufrufe • vor 9 Tagen

BOOM 🚨: Tencent’s Hunyuan Turbo S just landed in the Top 8 globally on Chatbot Arena and is now #2 in China, just behind DeepSeek. Even Google ex-CEO Eric Schmidt says China had no foundation models two years ago — now it has three: DeepSeek, Qwen, Hunyuan — on par with OpenAI’s O.1. At #GoogleIO, all three scored high on the global leaderboard like #DeepSeek, #Qwen, #Hunyuan Tencent Hy isn’t just racing — it’s rewriting the leaderboard. #HunyuanTurboS #TencentAI #LLM #AIRevolution
0:18

BOOM 🚨: Tencent’s Hunyuan Turbo S just landed in the Top 8 globally on Chatbot Arena and is now #2 in China, just behind DeepSeek. Even Google ex-CEO Eric Schmidt says China had no foundation models two years ago — now it has three: DeepSeek, Qwen, Hunyuan — on par with OpenAI’s O.1. At #GoogleIO, all three scored high on the global leaderboard like #DeepSeek, #Qwen, #Hunyuan Tencent Hy isn’t just racing — it’s rewriting the leaderboard. #HunyuanTurboS #TencentAI #LLM #AIRevolution

Emily Watson | AI Tools & Tech News

24,814 Aufrufe • vor 1 Jahr

Qwen3.7-Max on AI/ML API - built for the agent era GPQA Diamond (92.4), HMMT (97.1), Apex (44.5) Sustains 35+ hours of autonomous execution Works with Claude Code, Qwen Code & more Comment Qwen to get Free promo code
0:19

Qwen3.7-Max on AI/ML API - built for the agent era GPQA Diamond (92.4), HMMT (97.1), Apex (44.5) Sustains 35+ hours of autonomous execution Works with Claude Code, Qwen Code & more Comment Qwen to get Free promo code

AI/ML API

35,288 Aufrufe • vor 1 Monat

pip install spectralquant ✂️ Up to 6.62x KV cache compression for LLMs and transformers. Same model. Faster outputs. Smaller KV cache. Try now (2 mins): - KV cache integration via Hugging Face's DynamicCache - Three presets: 5.95x (paper), 6.55x (validated), 6.68x (edge) - Mistral 7B / Qwen 2.5 7B / Llama 3.1 8B verified - Pure PyTorch + future CUDA kernel support - Auto-calibration from a bundled corpus 📰 Paper: 💻 Code, quickstart, and benchmarks: #LLM #Inference #PyTorch #OpenSource #MachineLearning #LLM #KVCache #Inference
0:15

pip install spectralquant ✂️ Up to 6.62x KV cache compression for LLMs and transformers. Same model. Faster outputs. Smaller KV cache. Try now (2 mins): - KV cache integration via Hugging Face's DynamicCache - Three presets: 5.95x (paper), 6.55x (validated), 6.68x (edge) - Mistral 7B / Qwen 2.5 7B / Llama 3.1 8B verified - Pure PyTorch + future CUDA kernel support - Auto-calibration from a bundled corpus 📰 Paper: 💻 Code, quickstart, and benchmarks: #LLM #Inference #PyTorch #OpenSource #MachineLearning #LLM #KVCache #Inference

ani

16,583 Aufrufe • vor 29 Tagen

Someone got DeepSeek-R1-0528-Qwen3-8B running on an iPhone 16 Pro using MLX. It runs but takes ages to respond, and the phone gets hot fast. 8B models on phones aren't sci-fi anymore. via u/adrgrondin on r/LocalLLaMA
0:16

Someone got DeepSeek-R1-0528-Qwen3-8B running on an iPhone 16 Pro using MLX. It runs but takes ages to respond, and the phone gets hot fast. 8B models on phones aren't sci-fi anymore. via u/adrgrondin on r/LocalLLaMA

👋 Jan

82,742 Aufrufe • vor 1 Jahr

WIP Thinking UI for Qwen 3 thinking mode, running locally on iPhone. Tried to have something similar to Grok web thinking UI. Made 100% with SwiftUI.
0:12

WIP Thinking UI for Qwen 3 thinking mode, running locally on iPhone. Tried to have something similar to Grok web thinking UI. Made 100% with SwiftUI.

Adrien Grondin

64,271 Aufrufe • vor 1 Jahr

just 14 runs conceded in his last 2 overs with 8 yorkers, when England needed 69 off the last 5
0:11

just 14 runs conceded in his last 2 overs with 8 yorkers, when England needed 69 off the last 5

soo washed

48,425 Aufrufe • vor 3 Monaten