Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

I’ve been using the new MacBook Pro M5 since launch and it’s a beast, especially compared to my old M1! It’s good for running the smaller AI models locally but also everything you do is just so fast. Here’s Gemma 3 QAT 12B running with MLX:

Adrien Grondin

5,668 subscribers

101,950 views • 6 months ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

🔴 The Pain: Running local MLX models is incredibly fast and private. But let's be real - testing tool calling via terminal is clunky, and there's zero good UI for it. 🟢 The Fix: rapid-mlx share Just ONE command gives you a polished web chat + seamless tool calling (works beautifully with gemma-4-12b-qat). We are proud to be the ONLY MLX inference engine in the community shipping this. ⚡️ 👇 Try it now: brew install raullenchai/rapid-mlx/rapid-mlx

🔴 The Pain: Running local MLX models is incredibly fast and private. But let's be real - testing tool calling via terminal is clunky, and there's zero good UI for it. 🟢 The Fix: rapid-mlx share Just ONE command gives you a polished web chat + seamless tool calling (works beautifully with gemma-4-12b-qat). We are proud to be the ONLY MLX inference engine in the community shipping this. ⚡️ 👇 Try it now: brew install raullenchai/rapid-mlx/rapid-mlx

raullen

22,680 views • 19 days ago

Subagents running locally and simultaneously on MacBook Pro M5 with Codex CLI + LM Studio to review code and find bugs using Qwen 3.6 Powered by the updated MLX engine with batching in beta in the app The batching speed boost is noticeable

Subagents running locally and simultaneously on MacBook Pro M5 with Codex CLI + LM Studio to review code and find bugs using Qwen 3.6 Powered by the updated MLX engine with batching in beta in the app The batching speed boost is noticeable

Adrien Grondin

74,729 views • 1 month ago

This actually makes Gemma 4 26B-4A usable for a coding agent @ 72tk/s on my MacBook Pro M1 Max. This video is realtime, running completely locally.

This actually makes Gemma 4 26B-4A usable for a coding agent @ 72tk/s on my MacBook Pro M1 Max. This video is realtime, running completely locally.

Kyle Howells

117,577 views • 14 days ago

This afternoon I picked up a new Nvidia DGX Spark computer with the goal of trying to run Gemma 4 31b (4bit) on it locally as a server. Just 1.5 hours later, it’s working! Using Open WebUI on my MacBook as the interface and it’s connecting to my DGX Spark running as a Gemma 4 server.

This afternoon I picked up a new Nvidia DGX Spark computer with the goal of trying to run Gemma 4 31b (4bit) on it locally as a server. Just 1.5 hours later, it’s working! Using Open WebUI on my MacBook as the interface and it’s connecting to my DGX Spark running as a Gemma 4 server.

Dave Lee

137,810 views • 2 months ago

🔥 Thrilled to have worked with Google AI Developers on day-0 MLX support for Gemma 3 QAT! QAT optimizes models during training by simulating low-precision operations, delivering similar performance to FP16 and dramatic memory savings when quantised: • Gemma 3 27B: 54GB → 14.1GB (74% reduction) • Gemma 3 12B: 24GB → 6.6GB (72% reduction) • Gemma 3 4B: 8GB → 2.6GB (67% reduction) • Gemma 3 1B: 2GB → 0.5GB (75% reduction) Get started: > pip install mlx-vlm mlx-lm Model collection 👇🏽

🔥 Thrilled to have worked with Google AI Developers on day-0 MLX support for Gemma 3 QAT! QAT optimizes models during training by simulating low-precision operations, delivering similar performance to FP16 and dramatic memory savings when quantised: • Gemma 3 27B: 54GB → 14.1GB (74% reduction) • Gemma 3 12B: 24GB → 6.6GB (72% reduction) • Gemma 3 4B: 8GB → 2.6GB (67% reduction) • Gemma 3 1B: 2GB → 0.5GB (75% reduction) Get started: > pip install mlx-vlm mlx-lm Model collection 👇🏽

Prince Canuma

11,374 views • 1 year ago

ParoQuant just got a big upgrade 🚀 ✅ Supports the new Qwen3.5 models ⚡ Now runs on MLX (fast local inference on Apple Silicon) 🧠 Preserves reasoning quality with 4-bit quantization We also built an agent demo running locally on my 4-year-old M2 Max. Can't wait to upgrade to an M5 Max and see what kind of magic we can do. ✨

ParoQuant just got a big upgrade 🚀 ✅ Supports the new Qwen3.5 models ⚡ Now runs on MLX (fast local inference on Apple Silicon) 🧠 Preserves reasoning quality with 4-bit quantization We also built an agent demo running locally on my 4-year-old M2 Max. Can't wait to upgrade to an M5 Max and see what kind of magic we can do. ✨

Zhijian Liu

49,004 views • 3 months ago

This video is at normal speed. Gemma 4 12B MLX version running locally at 50 tokens/sec. Thank you Google DeepMind team. This model feels really solid for a lot of small local tasks and everyday AI workflows.

This video is at normal speed. Gemma 4 12B MLX version running locally at 50 tokens/sec. Thank you Google DeepMind team. This model feels really solid for a lot of small local tasks and everyday AI workflows.

AshutoshShrivastava

19,782 views • 21 days ago

Running Gemma 4 12B locally at 28T/s. Facing issues with vision in LM Studio, but text performance is quite solid. Asked it to create a landing page for a tech event. Video is fast forwarded for the demo.

Running Gemma 4 12B locally at 28T/s. Facing issues with vision in LM Studio, but text performance is quite solid. Asked it to create a landing page for a tech event. Video is fast forwarded for the demo.

AshutoshShrivastava

15,680 views • 22 days ago

I tested Gemma 4 12b on my Macbook Pro M5 Max with 128GB of unified memory. It runs VERY slow for a 12b parameter model. 44 tokens/second. The lava lamp that it produced was also terrible. I would stick to Qwen 3.6 27b or Qwen 3.6 35b. Google is still way behind, even with their local models.

I tested Gemma 4 12b on my Macbook Pro M5 Max with 128GB of unified memory. It runs VERY slow for a 12b parameter model. 44 tokens/second. The lava lamp that it produced was also terrible. I would stick to Qwen 3.6 27b or Qwen 3.6 35b. Google is still way behind, even with their local models.

BridgeMind

70,094 views • 20 days ago

I'm exploring running LLMs locally on iPhones and Macs. I’ve got Llama 3.2 running locally using Apple’s MLX and with support for tool calling. This example runs two LLMs: one to identify tools to call based on the query and one to generate responses based on the tools’ outputs.

I'm exploring running LLMs locally on iPhones and Macs. I’ve got Llama 3.2 running locally using Apple’s MLX and with support for tool calling. This example runs two LLMs: one to identify tools to call based on the query and one to generate responses based on the tools’ outputs.

Simon B. Støvring

47,972 views • 1 year ago

I got a fully maxed out MacBook, mostly so I can run local models fast. And omg here is mixtral running on ollama... almost cannot believe how fast it is. This is with no internet!! A model that beats GPT-3.5 running locally! What!

I got a fully maxed out MacBook, mostly so I can run local models fast. And omg here is mixtral running on ollama... almost cannot believe how fast it is. This is with no internet!! A model that beats GPT-3.5 running locally! What!

Charlie Holtz

196,035 views • 2 years ago

The new MacBook Pro M4 is crazy fast. Loving this upgrade. What can you see yourself using this beast to do?🔥💻 #MacBookPro #AppleM4 #TechUpgrade

Sensitive content

The new MacBook Pro M4 is crazy fast. Loving this upgrade. What can you see yourself using this beast to do?🔥💻 #MacBookPro #AppleM4 #TechUpgrade

UrAvgConsumer

14,241 views • 1 year ago

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.

kwindla

202,113 views • 10 months ago

Run the latest Ministral 3 models by Mistral AI on macOS. Running privately on-device and optimized for Apple Silicon with MLX.

Run the latest Ministral 3 models by Mistral AI on macOS. Running privately on-device and optimized for Apple Silicon with MLX.

Locally AI - Local AI Chat

34,901 views • 6 months ago

In-Browser-AI-Models are here 🤯 This is Gemini Nano running on my M1 Macbook Pro using the latest Chrome Canary version. Most of the text generations are completed in under 500ms - entirely offline too! Adding this to a Vue app is only ~15 lines of code. Pretty insane!

In-Browser-AI-Models are here 🤯 This is Gemini Nano running on my M1 Macbook Pro using the latest Chrome Canary version. Most of the text generations are completed in under 500ms - entirely offline too! Adding this to a Vue app is only ~15 lines of code. Pretty insane!

Marcel Pociot 🧪

26,335 views • 2 years ago

people think running AI locally requires: → $3,000 MacBook Pro → RTX 4090 → $20/month cloud subscription nvidia just dropped a $249 computer. 67 TOPS. runs llama 3.1-8B locally. no internet. no API. no monthly fee. ever. smaller than your router. costs the same as AirPods. runs the same models you pay $240/year to access via ChatGPT. the local AI era just got a price tag. $249.

people think running AI locally requires: → $3,000 MacBook Pro → RTX 4090 → $20/month cloud subscription nvidia just dropped a $249 computer. 67 TOPS. runs llama 3.1-8B locally. no internet. no API. no monthly fee. ever. smaller than your router. costs the same as AirPods. runs the same models you pay $240/year to access via ChatGPT. the local AI era just got a price tag. $249.

self.dll

320,926 views • 1 month ago

Pretty cool that with the new Qwen 2.5 models you can ask questions / generate using a reasonably sized code-base as context, all running on a laptop with mlx-lm. The 7B runs pretty fast on an M4 Max using the mlx-lm code base (~16k lines) as context:

Pretty cool that with the new Qwen 2.5 models you can ask questions / generate using a reasonably sized code-base as context, all running on a laptop with mlx-lm. The 7B runs pretty fast on an M4 Max using the mlx-lm code base (~16k lines) as context:

Awni Hannun

27,442 views • 1 year ago

Running GLM-4.7-Flash with OpenCode locally on M4 Max MacBook Pro. 4-bit model runs at 82 tok/sec. Prefill will get ~4x faster with M5 Max MacBook Pro (~28 Jan). EXO will also support disaggregating prefill and decode across devices, e.g. DGX Spark.

Running GLM-4.7-Flash with OpenCode locally on M4 Max MacBook Pro. 4-bit model runs at 82 tok/sec. Prefill will get ~4x faster with M5 Max MacBook Pro (~28 Jan). EXO will also support disaggregating prefill and decode across devices, e.g. DGX Spark.

Alex Cheema

127,902 views • 5 months ago

The P211 GTO Is here! And yes, it takes 320 Mags! I feel like I’ve been waiting 17 years for this but I’m thrilled to announce that it’s officially on its way to you. This of course is extra special for me since I’ve been running this platform since I was 15 years old.

The P211 GTO Is here! And yes, it takes 320 Mags! I feel like I’ve been waiting 17 years for this but I’m thrilled to announce that it’s officially on its way to you. This of course is extra special for me since I’ve been running this platform since I was 15 years old.

Max Michel

15,917 views • 1 year ago

It’s been a while since my last post, but I just wanted to say that the Cropdeck launch went pretty well! I’ve been busy preparing patches for critical bugs and balance issues, but it’s getting calmer now 😁

It’s been a while since my last post, but I just wanted to say that the Cropdeck launch went pretty well! I’ve been busy preparing patches for critical bugs and balance issues, but it’s getting calmer now 😁

Piotrek - Wishlist Cropdeck 🌱 on Steam!

11,106 views • 1 month ago