Загрузка видео...

Не удалось загрузить видео

На главную

MLX Swift LLM example works with: - Mistral / Llama - Phi-2 - Qwen 1.5 - Starcoder 2 Quick-start: Qwen 1.5 0.5B runs pretty fast in 16-bit on my iPhone 14, no quantization needed:

awnihannun's profile picture

Awni Hannun

44,814 subscribers

30,441 просмотров • 2 лет назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

DeepSeek R1 distilled to Qwen 1.5B easily runs on my iPhone 16 with MLX swift. Here's the 4-bit model reasoning entirely on device at almost 60 toks/sec:
0:19

DeepSeek R1 distilled to Qwen 1.5B easily runs on my iPhone 16 with MLX swift. Here's the 4-bit model reasoning entirely on device at almost 60 toks/sec:

Awni Hannun

1,139,656 просмотров • 1 год назад

Helium 2B running locally on an iPhone 16 Pro at 28 tok/s, faster than you can read, all thanks to mlx-swift with q4 quantization!
0:13

Helium 2B running locally on an iPhone 16 Pro at 28 tok/s, faster than you can read, all thanks to mlx-swift with q4 quantization!

kyutai

59,240 просмотров • 1 год назад

I was exploring MLX Swift and created a workout assistant to train for my runs! 🏃🏽‍♂️ It uses my workout data from HealthKit for personalization and Qwen 2.5 3B for local inference!
0:13

I was exploring MLX Swift and created a workout assistant to train for my runs! 🏃🏽‍♂️ It uses my workout data from HealthKit for personalization and Qwen 2.5 3B for local inference!

Rudrank Riyam

19,965 просмотров • 1 год назад

Qwen 3.5 0.8B, Gated DeltaNet attention is running on Apple Neural Engine ~56 t/s in LUT6 quantization with some room for optimization left. It is CoreML, Swift and IOSurface on M4Pro. It will slow down as we increase context, but not by much. I think Private API opens the way to integrate ANE with GPU/MLX and possibly some MoE.
0:19

Qwen 3.5 0.8B, Gated DeltaNet attention is running on Apple Neural Engine ~56 t/s in LUT6 quantization with some room for optimization left. It is CoreML, Swift and IOSurface on M4Pro. It will slow down as we increase context, but not by much. I think Private API opens the way to integrate ANE with GPU/MLX and possibly some MoE.

Anemll

13,589 просмотров • 3 месяцев назад

Another demo of the iPhone 17 Pro’s on-device LLM performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quants
0:11

Another demo of the iPhone 17 Pro’s on-device LLM performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quants

Adrien Grondin

46,205 просмотров • 8 месяцев назад

DeepSeek-R1-0528-Qwen3-8B running on iPhone 16 with MLX
0:16

DeepSeek-R1-0528-Qwen3-8B running on iPhone 16 with MLX

ℏεsam

17,285 просмотров • 1 год назад

Qwen 3 family is shipping with the upcoming 0.4.0 version of React Native ExecuTorch. 🎉 The 1.7B variant reaches 30 tokens per second on iPhone 16 Pro!
0:18

Qwen 3 family is shipping with the upcoming 0.4.0 version of React Native ExecuTorch. 🎉 The 1.7B variant reaches 30 tokens per second on iPhone 16 Pro!

Software Mansion

11,637 просмотров • 1 год назад

Managed to get Ling Mini 16B (1.4B active) running on my iPhone Air. It runs very fast with MLX. It's a DWQ of Ling Mini quantized to 3 bits-per-weight. A 16B model running on an Air at this speed is pretty awesome:
0:16

Managed to get Ling Mini 16B (1.4B active) running on my iPhone Air. It runs very fast with MLX. It's a DWQ of Ling Mini quantized to 3 bits-per-weight. A 16B model running on an Air at this speed is pretty awesome:

Awni Hannun

92,422 просмотров • 8 месяцев назад

DeepSeek R1 Qwen 7B 4bit M2 Ultra vs M4 Max on Apple MLX 🤫 Let them think... (video 4x in center part) M2 Ultra: 114.9 tokens per sec M4 Max (14"): 88.3 tokens per sec
0:16

DeepSeek R1 Qwen 7B 4bit M2 Ultra vs M4 Max on Apple MLX 🤫 Let them think... (video 4x in center part) M2 Ultra: 114.9 tokens per sec M4 Max (14"): 88.3 tokens per sec

Ivan Fioravanti ᯅ

59,734 просмотров • 1 год назад

Zuckerberg watching you use Qwen instead of LLaMA > via /r/rlocallama
0:11

Zuckerberg watching you use Qwen instead of LLaMA > via /r/rlocallama

Chubby♨️

300,751 просмотров • 1 год назад

Here Qwen 3 0.6B again on Italian wines 82.5%! The best commercial LLM I tested in the past was scoring 75.6% (Sonnet 3.5)
0:19

Here Qwen 3 0.6B again on Italian wines 82.5%! The best commercial LLM I tested in the past was scoring 75.6% (Sonnet 3.5)

Ivan Fioravanti ᯅ

25,367 просмотров • 1 год назад

BOOM 🚨: Tencent’s Hunyuan Turbo S just landed in the Top 8 globally on Chatbot Arena and is now #2 in China, just behind DeepSeek. Even Google ex-CEO Eric Schmidt says China had no foundation models two years ago — now it has three: DeepSeek, Qwen, Hunyuan — on par with OpenAI’s O.1. At #GoogleIO, all three scored high on the global leaderboard like #DeepSeek, #Qwen, #Hunyuan Tencent Hy isn’t just racing — it’s rewriting the leaderboard. #HunyuanTurboS #TencentAI #LLM #AIRevolution
0:18

BOOM 🚨: Tencent’s Hunyuan Turbo S just landed in the Top 8 globally on Chatbot Arena and is now #2 in China, just behind DeepSeek. Even Google ex-CEO Eric Schmidt says China had no foundation models two years ago — now it has three: DeepSeek, Qwen, Hunyuan — on par with OpenAI’s O.1. At #GoogleIO, all three scored high on the global leaderboard like #DeepSeek, #Qwen, #Hunyuan Tencent Hy isn’t just racing — it’s rewriting the leaderboard. #HunyuanTurboS #TencentAI #LLM #AIRevolution

Emily Watson | AI Tools & Tech News

24,814 просмотров • 1 год назад

Qwen3.7-Max on AI/ML API - built for the agent era GPQA Diamond (92.4), HMMT (97.1), Apex (44.5) Sustains 35+ hours of autonomous execution Works with Claude Code, Qwen Code & more Comment Qwen to get Free promo code
0:19

Qwen3.7-Max on AI/ML API - built for the agent era GPQA Diamond (92.4), HMMT (97.1), Apex (44.5) Sustains 35+ hours of autonomous execution Works with Claude Code, Qwen Code & more Comment Qwen to get Free promo code

AI/ML API

35,210 просмотров • 26 дней назад

Someone got DeepSeek-R1-0528-Qwen3-8B running on an iPhone 16 Pro using MLX. It runs but takes ages to respond, and the phone gets hot fast. 8B models on phones aren't sci-fi anymore. via u/adrgrondin on r/LocalLLaMA
0:16

Someone got DeepSeek-R1-0528-Qwen3-8B running on an iPhone 16 Pro using MLX. It runs but takes ages to respond, and the phone gets hot fast. 8B models on phones aren't sci-fi anymore. via u/adrgrondin on r/LocalLLaMA

👋 Jan

82,742 просмотров • 1 год назад

WIP Thinking UI for Qwen 3 thinking mode, running locally on iPhone. Tried to have something similar to Grok web thinking UI. Made 100% with SwiftUI.
0:12

WIP Thinking UI for Qwen 3 thinking mode, running locally on iPhone. Tried to have something similar to Grok web thinking UI. Made 100% with SwiftUI.

Adrien Grondin

64,271 просмотров • 1 год назад

just 14 runs conceded in his last 2 overs with 8 yorkers, when England needed 69 off the last 5
0:11

just 14 runs conceded in his last 2 overs with 8 yorkers, when England needed 69 off the last 5

soo washed

48,349 просмотров • 3 месяцев назад

Offline AI, right on your phone. Small versions of Hermes 3, Qwen 2.5, and Llama 3.2 running privately in your hand. Grab them now in Apollo 1.0.17
0:18

Offline AI, right on your phone. Small versions of Hermes 3, Qwen 2.5, and Llama 3.2 running privately in your hand. Grab them now in Apollo 1.0.17

Aaron Ng

36,884 просмотров • 1 год назад

Running the full GLM 4.7 (8-bit) on 2 x 512GB M3 Ultra Mac Studios Runs at 19.8 tok/sec with EXO Labs MLX RDMA backend (h/t Awni Hannun) & tensor parallel
0:12

Running the full GLM 4.7 (8-bit) on 2 x 512GB M3 Ultra Mac Studios Runs at 19.8 tok/sec with EXO Labs MLX RDMA backend (h/t Awni Hannun) & tensor parallel

Alex Cheema

156,668 просмотров • 5 месяцев назад