Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

I’ve been using the new MacBook Pro M5 since launch and it’s a beast, especially compared to my old M1! It’s good for running the smaller AI models locally but also everything you do is just so fast. Here’s Gemma 3 QAT 12B running with MLX:

adrgrondin's profile picture

Adrien Grondin

5,668 subscribers

101,950 Aufrufe • vor 11 Monaten •via X (Twitter)

BildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

🔴 The Pain: Running local MLX models is incredibly fast and private. But let's be real - testing tool calling via terminal is clunky, and there's zero good UI for it. 🟢 The Fix: rapid-mlx share Just ONE command gives you a polished web chat + seamless tool calling (works beautifully with gemma-4-12b-qat). We are proud to be the ONLY MLX inference engine in the community shipping this. ⚡️ 👇 Try it now: brew install raullenchai/rapid-mlx/rapid-mlx
0:40

🔴 The Pain: Running local MLX models is incredibly fast and private. But let's be real - testing tool calling via terminal is clunky, and there's zero good UI for it. 🟢 The Fix: rapid-mlx share Just ONE command gives you a polished web chat + seamless tool calling (works beautifully with gemma-4-12b-qat). We are proud to be the ONLY MLX inference engine in the community shipping this. ⚡️ 👇 Try it now: brew install raullenchai/rapid-mlx/rapid-mlx

raullen

22,776 Aufrufe • vor 5 Monaten

Subagents running locally and simultaneously on MacBook Pro M5 with Codex CLI + LM Studio to review code and find bugs using Qwen 3.6 Powered by the updated MLX engine with batching in beta in the app The batching speed boost is noticeable
1:40

Subagents running locally and simultaneously on MacBook Pro M5 with Codex CLI + LM Studio to review code and find bugs using Qwen 3.6 Powered by the updated MLX engine with batching in beta in the app The batching speed boost is noticeable

Adrien Grondin

75,549 Aufrufe • vor 5 Monaten

This afternoon I picked up a new Nvidia DGX Spark computer with the goal of trying to run Gemma 4 31b (4bit) on it locally as a server. Just 1.5 hours later, it’s working! Using Open WebUI on my MacBook as the interface and it’s connecting to my DGX Spark running as a Gemma 4 server.
0:46

This afternoon I picked up a new Nvidia DGX Spark computer with the goal of trying to run Gemma 4 31b (4bit) on it locally as a server. Just 1.5 hours later, it’s working! Using Open WebUI on my MacBook as the interface and it’s connecting to my DGX Spark running as a Gemma 4 server.

Dave Lee

141,105 Aufrufe • vor 7 Monaten

This actually makes Gemma 4 26B-4A usable for a coding agent @ 72tk/s on my MacBook Pro M1 Max. This video is realtime, running completely locally.
0:57

This actually makes Gemma 4 26B-4A usable for a coding agent @ 72tk/s on my MacBook Pro M1 Max. This video is realtime, running completely locally.

Kyle Howells

119,560 Aufrufe • vor 5 Monaten

ParoQuant just got a big upgrade 🚀 ✅ Supports the new Qwen3.5 models ⚡ Now runs on MLX (fast local inference on Apple Silicon) 🧠 Preserves reasoning quality with 4-bit quantization We also built an agent demo running locally on my 4-year-old M2 Max. Can't wait to upgrade to an M5 Max and see what kind of magic we can do. ✨
0:46

ParoQuant just got a big upgrade 🚀 ✅ Supports the new Qwen3.5 models ⚡ Now runs on MLX (fast local inference on Apple Silicon) 🧠 Preserves reasoning quality with 4-bit quantization We also built an agent demo running locally on my 4-year-old M2 Max. Can't wait to upgrade to an M5 Max and see what kind of magic we can do. ✨

Zhijian Liu

49,826 Aufrufe • vor 8 Monaten

This video is at normal speed. Gemma 4 12B MLX version running locally at 50 tokens/sec. Thank you Google DeepMind team. This model feels really solid for a lot of small local tasks and everyday AI workflows.
0:50

This video is at normal speed. Gemma 4 12B MLX version running locally at 50 tokens/sec. Thank you Google DeepMind team. This model feels really solid for a lot of small local tasks and everyday AI workflows.

AshutoshShrivastava

19,853 Aufrufe • vor 5 Monaten

Running Gemma 4 12B locally at 28T/s. Facing issues with vision in LM Studio, but text performance is quite solid. Asked it to create a landing page for a tech event. Video is fast forwarded for the demo.
0:40

Running Gemma 4 12B locally at 28T/s. Facing issues with vision in LM Studio, but text performance is quite solid. Asked it to create a landing page for a tech event. Video is fast forwarded for the demo.

AshutoshShrivastava

15,697 Aufrufe • vor 5 Monaten

I tested Gemma 4 12b on my Macbook Pro M5 Max with 128GB of unified memory. It runs VERY slow for a 12b parameter model. 44 tokens/second. The lava lamp that it produced was also terrible. I would stick to Qwen 3.6 27b or Qwen 3.6 35b. Google is still way behind, even with their local models.
1:36

I tested Gemma 4 12b on my Macbook Pro M5 Max with 128GB of unified memory. It runs VERY slow for a 12b parameter model. 44 tokens/second. The lava lamp that it produced was also terrible. I would stick to Qwen 3.6 27b or Qwen 3.6 35b. Google is still way behind, even with their local models.

BridgeMind

70,626 Aufrufe • vor 5 Monaten

Qwen 3.8 27B doing an agentic task with subagent at ~120 tok/s on M5 Max MacBook Pro in LM Studio Bionic using Inco AI Splash engine At launch a little more than a month ago the model was running around ~20tk/s It’s 6x faster now, local AI is progressing at lightning speed!
0:45

Qwen 3.8 27B doing an agentic task with subagent at ~120 tok/s on M5 Max MacBook Pro in LM Studio Bionic using Inco AI Splash engine At launch a little more than a month ago the model was running around ~20tk/s It’s 6x faster now, local AI is progressing at lightning speed!

Adrien Grondin

47,846 Aufrufe • vor 1 Monat

I got a fully maxed out MacBook, mostly so I can run local models fast. And omg here is mixtral running on ollama... almost cannot believe how fast it is. This is with no internet!! A model that beats GPT-3.5 running locally! What!
0:20

I got a fully maxed out MacBook, mostly so I can run local models fast. And omg here is mixtral running on ollama... almost cannot believe how fast it is. This is with no internet!! A model that beats GPT-3.5 running locally! What!

Charlie Holtz

196,086 Aufrufe • vor 2 Jahren

The new MacBook Pro M4 is crazy fast. Loving this upgrade. What can you see yourself using this beast to do?🔥💻 #MacBookPro #AppleM4 #TechUpgrade
0:59

Sensitive content

The new MacBook Pro M4 is crazy fast. Loving this upgrade. What can you see yourself using this beast to do?🔥💻 #MacBookPro #AppleM4 #TechUpgrade

UrAvgConsumer

14,464 Aufrufe • vor 1 Jahr

The new MacBook Pro, with tandem oled and M5 Pro or M5 Max
1:49

The new MacBook Pro, with tandem oled and M5 Pro or M5 Max

Tyler Stalman

77,521 Aufrufe • vor 13 Tagen

MLX: Tom Riddle's diary running locally on an iPad powered by Apple MLX with Gemma 4 E4B behind the scene. Children can go crazy for something like this, no? 🔥
2:49

MLX: Tom Riddle's diary running locally on an iPad powered by Apple MLX with Gemma 4 E4B behind the scene. Children can go crazy for something like this, no? 🔥

Ivan Fioravanti

32,511 Aufrufe • vor 4 Monaten

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.
4:35

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.

kwindla

202,309 Aufrufe • vor 1 Jahr

Here’s Half-Life: Alyx running natively on my Apple Vision Pro M5. No PC, no streaming. It’s pretty incredible.
1:39

Here’s Half-Life: Alyx running natively on my Apple Vision Pro M5. No PC, no streaming. It’s pretty incredible.

Justin Ryan

36,189 Aufrufe • vor 1 Monat

The top question I got was does Halo support local LLM running on a Mac. Here's an example of the app using gemma-4-26b-a4b-qat via LM Studio running on a base M5/32GB. It's giving an average of 40 tok/s and in my brief testing is able to execute all tool calls correctly. Very usable with this app.
0:37

The top question I got was does Halo support local LLM running on a Mac. Here's an example of the app using gemma-4-26b-a4b-qat via LM Studio running on a base M5/32GB. It's giving an average of 40 tok/s and in my brief testing is able to execute all tool calls correctly. Very usable with this app.

Tanmay

11,682 Aufrufe • vor 4 Monaten

The P211 GTO Is here! And yes, it takes 320 Mags! I feel like I’ve been waiting 17 years for this but I’m thrilled to announce that it’s officially on its way to you. This of course is extra special for me since I’ve been running this platform since I was 15 years old.
2:34

The P211 GTO Is here! And yes, it takes 320 Mags! I feel like I’ve been waiting 17 years for this but I’m thrilled to announce that it’s officially on its way to you. This of course is extra special for me since I’ve been running this platform since I was 15 years old.

Max Michel

15,917 Aufrufe • vor 1 Jahr

72+ TPS on Qwen 3.6 27B on a Macbook pro M5 max. MTPLX V2 out now! The fastest way to run models on MLX.
0:26

72+ TPS on Qwen 3.6 27B on a Macbook pro M5 max. MTPLX V2 out now! The fastest way to run models on MLX.

Youssof Al Toukhi

91,865 Aufrufe • vor 4 Monaten