Loading video...

Video Failed to Load

Go Home

GLM-4.7 Flash by Z.ai running on M3 Ultra using MLX! 4bit at 81 toks/s 🔥 8bit at 64 toks/s 🔥 (video below) Model converted from transformers to MLX on the fly with Codex + GPT-5.2 HIgh!

ivanfioravanti's profile picture

Ivan Fioravanti ᯅ

30,583 subscribers

75,425 views • 7 months ago •via X (Twitter)

EducationScience & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

MLX Step-3.5-Flash! I've reached 45 toks/s! Thanks to Fast-MLX skill by Awni Hannun used withing Codex with GPT 5.2 High! From 13 toks/s v0 to 45 toks/s v2 🚀 but again I bet Tarjei Mandt will do even better 🙌🏻
0:12

MLX Step-3.5-Flash! I've reached 45 toks/s! Thanks to Fast-MLX skill by Awni Hannun used withing Codex with GPT 5.2 High! From 13 toks/s v0 to 45 toks/s v2 🚀 but again I bet Tarjei Mandt will do even better 🙌🏻

Ivan Fioravanti ᯅ

13,933 views • 6 months ago

DeepSeek-Prover (4-bit 7B) running at 114 toks/sec in MLX LM on an M2 Ultra Thanks to for the port!
0:16

DeepSeek-Prover (4-bit 7B) running at 114 toks/sec in MLX LM on an M2 Ultra Thanks to for the port!

Awni Hannun

16,079 views • 2 years ago

Qwen QwQ 32B fp16 on M4 Max and M2 Ultra powered by MLX! M2 Ultra - 10.2 toks/s M4 Max - 7.6 toks/s! "Create an amazing animation using p5js" o1-mini level local model! Note: use temp 0.7-0.75 for optimal results in coding. I did some tests and this
0:10

Qwen QwQ 32B fp16 on M4 Max and M2 Ultra powered by MLX! M2 Ultra - 10.2 toks/s M4 Max - 7.6 toks/s! "Create an amazing animation using p5js" o1-mini level local model! Note: use temp 0.7-0.75 for optimal results in coding. I did some tests and this

Ivan Fioravanti ᯅ

62,377 views • 1 year ago

Running the full GLM 4.7 (8-bit) on 2 x 512GB M3 Ultra Mac Studios Runs at 19.8 tok/sec with EXO Labs MLX RDMA backend (h/t Awni Hannun) & tensor parallel
0:12

Running the full GLM 4.7 (8-bit) on 2 x 512GB M3 Ultra Mac Studios Runs at 19.8 tok/sec with EXO Labs MLX RDMA backend (h/t Awni Hannun) & tensor parallel

Alex Cheema

156,668 views • 7 months ago

GLM 5.2 has just been released 🔥 Here it's already running with MLX on two Mac Studios (M3 Ultra). This is comparable to the latest closed models, with weights you can download, quantize, distill, fine-tune, run.
0:11

GLM 5.2 has just been released 🔥 Here it's already running with MLX on two Mac Studios (M3 Ultra). This is comparable to the latest closed models, with weights you can download, quantize, distill, fine-tune, run.

Pedro Cuenca

83,401 views • 2 months ago

MiniMax M2.5 casually responding at ~50 tok/s with MLX (M3 Ultra). The model was released one hour ago 🥳
0:14

MiniMax M2.5 casually responding at ~50 tok/s with MLX (M3 Ultra). The model was released one hour ago 🥳

Pedro Cuenca

170,567 views • 6 months ago

Playing with Qwen3-TTS is and MLX-Audio locally on Mac Studio M3 Ultra 🔥 Amazing model by Qwen and great work by Prince Canuma bringing this magic to MLX! Tuning the voice with instructions feels like magic! Command and prompts to run it are in the video.
0:17

Playing with Qwen3-TTS is and MLX-Audio locally on Mac Studio M3 Ultra 🔥 Amazing model by Qwen and great work by Prince Canuma bringing this magic to MLX! Tuning the voice with instructions feels like magic! Command and prompts to run it are in the video.

Ivan Fioravanti ᯅ

35,720 views • 6 months ago

Running Kimi K2.5 on my desk. Runs at 24 tok/sec with 2 x 512GB M3 Ultra Mac Studios connected with Thunderbolt 5 (RDMA) using EXO Labs / MLX backend. Yes, it can run clawdbot.
0:18

Running Kimi K2.5 on my desk. Runs at 24 tok/sec with 2 x 512GB M3 Ultra Mac Studios connected with Thunderbolt 5 (RDMA) using EXO Labs / MLX backend. Yes, it can run clawdbot.

Alex Cheema

3,300,794 views • 6 months ago

i found a way to make UNCENSORED AI AGENT on a RTX 4090 GPU (!!!) with LOCAL 30B model weights this is GLM-4.7-Flash with abliteration, need 24GB VRAM, safety alignment surgically removed from the weights, the model has native tool calling, it actually executes bash, edits files, runs git (1) use ollama to pull weights of GLM > ollama pull huihui_ai/glm-4.7-flash-abliterated:q4_K (2) proxy it to any coding agent via ollama > ollama launch claude --model huihui_ai/glm-4.7-flash-abliterated:q4_K > ollama launch codex --model huihui_ai/glm-4.7-flash-abliterated:q4_K > ollama launch opencode --model huihui_ai/glm-4.7-flash-abliterated:q4_K (3) have fun
0:11

i found a way to make UNCENSORED AI AGENT on a RTX 4090 GPU (!!!) with LOCAL 30B model weights this is GLM-4.7-Flash with abliteration, need 24GB VRAM, safety alignment surgically removed from the weights, the model has native tool calling, it actually executes bash, edits files, runs git (1) use ollama to pull weights of GLM > ollama pull huihui_ai/glm-4.7-flash-abliterated:q4_K (2) proxy it to any coding agent via ollama > ollama launch claude --model huihui_ai/glm-4.7-flash-abliterated:q4_K > ollama launch codex --model huihui_ai/glm-4.7-flash-abliterated:q4_K > ollama launch opencode --model huihui_ai/glm-4.7-flash-abliterated:q4_K (3) have fun

chiefofautism

341,710 views • 6 months ago

MLX MiniMax 2.5 running LOCALLY on a single M3 Ultra 512GB! Writing a poem on LLMs at 6bit quantization! 🔥 Let's start some coding, context and distributed tests! Generation: 40.2 tokens-per-sec Peak memory: 186 GB
0:11

MLX MiniMax 2.5 running LOCALLY on a single M3 Ultra 512GB! Writing a poem on LLMs at 6bit quantization! 🔥 Let's start some coding, context and distributed tests! Generation: 40.2 tokens-per-sec Peak memory: 186 GB

Ivan Fioravanti ᯅ

226,257 views • 6 months ago

Another demo of the iPhone 17 Pro’s on-device LLM performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quants
0:11

Another demo of the iPhone 17 Pro’s on-device LLM performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quants

Adrien Grondin

46,205 views • 10 months ago

Quick demo of LFM 2.5 VL 1.6B model by Liquid AI that I recently added to Locally AI - Local AI Chat Running locally on iPhone 17 Pro at ~90tk/s with MLX Small vision-language models are improving fast
0:10

Quick demo of LFM 2.5 VL 1.6B model by Liquid AI that I recently added to Locally AI - Local AI Chat Running locally on iPhone 17 Pro at ~90tk/s with MLX Small vision-language models are improving fast

Adrien Grondin

32,556 views • 7 months ago

Google’s Gemma 4 E2B running on-device on iPhone 17 Pro Gemma 4 is built from the same research as Gemini 3, has image understanding capabilities and can reason if needed Running at ~40tk/s with MLX optimized for Apple Silicon
0:12

Google’s Gemma 4 E2B running on-device on iPhone 17 Pro Gemma 4 is built from the same research as Gemini 3, has image understanding capabilities and can reason if needed Running at ~40tk/s with MLX optimized for Apple Silicon

Adrien Grondin

1,044,880 views • 4 months ago

Open weights just caught up to the frontier. GLM-5.2 from Z.ai tops the open-model rankings on Artificial Analysis and Arena.ai's Agent Arena. It's now live on CoreWeave Serverless Inference at $1.39 in and $4.40 out per 1M tokens. Ship more for less.
0:18

Open weights just caught up to the frontier. GLM-5.2 from Z.ai tops the open-model rankings on Artificial Analysis and Arena.ai's Agent Arena. It's now live on CoreWeave Serverless Inference at $1.39 in and $4.40 out per 1M tokens. Ship more for less.

CoreWeave

16,637 views • 1 month ago

Gemma 4 looks at a parking lot. Decides what to ask. Calls SAM 3.1. "Segment all vehicles." 64 found. "Now just the white ones." 23 found. One model reasoning and orchestrating. One model executing. Both running locally on a MacBook. MLX. No cloud. No API.
0:14

Gemma 4 looks at a parking lot. Decides what to ask. Calls SAM 3.1. "Segment all vehicles." 64 found. "Now just the white ones." 23 found. One model reasoning and orchestrating. One model executing. Both running locally on a MacBook. MLX. No cloud. No API.

Maziyar PANAHI

594,212 views • 4 months ago

Quick update on the water situation 💦 M3 Ultra and Titan (RTX6000 Pro) seem to have recovered with little to no visible damage. The main issues are with my MacBook which is in service and Titan CPU temperatures being above avg when idling (58C up from 35C prior to water incident). Anyways, here is a video of MLX-VLM serving Qwen3-4B-Instruct on Titan (~300 tok/s) to do autocomplete and git commit message generation completely locally via Zed IDE.
0:13

Quick update on the water situation 💦 M3 Ultra and Titan (RTX6000 Pro) seem to have recovered with little to no visible damage. The main issues are with my MacBook which is in service and Titan CPU temperatures being above avg when idling (58C up from 35C prior to water incident). Anyways, here is a video of MLX-VLM serving Qwen3-4B-Instruct on Titan (~300 tok/s) to do autocomplete and git commit message generation completely locally via Zed IDE.

Prince Canuma

17,065 views • 3 months ago

I tested MTPLX v2 with QWEN 3.6 27B and compared it with oMLX without cache on M5 Max and DGX Spark on vllm using nvfp4 model version. More details in 🧵 I've reached 82.8 tps of max decoding speed! 🔥 Custom Metal Kernel design specifically for this model and for Apple Silicon is just perfect! This is the way forward! Great job Youssof Al Toukhi Look at the website here! 👇 Here a website with recap, built with GLM 5.2 running locally 💪 First chart and preview from the website.
0:15

I tested MTPLX v2 with QWEN 3.6 27B and compared it with oMLX without cache on M5 Max and DGX Spark on vllm using nvfp4 model version. More details in 🧵 I've reached 82.8 tps of max decoding speed! 🔥 Custom Metal Kernel design specifically for this model and for Apple Silicon is just perfect! This is the way forward! Great job Youssof Al Toukhi Look at the website here! 👇 Here a website with recap, built with GLM 5.2 running locally 💪 First chart and preview from the website.

Ivan Fioravanti ᯅ

15,846 views • 1 month ago

i just tested to put GLM-5.2 on my rig. 753B parameter MoE. 2x RTX PRO 6000 Blackwells, Threadripper PRO 9995WX with 1TB DDR5. prefills at 64 tok/s. decode holds at 13-15. system RAM bandwidth is the bottleneck. running UD-Q4_K_XL 4-bit. 69.7% on Aider Polyglot, within two points of BF16. left the 5090 off. mismatched VRAM unbalances the split. this is what local AI looks like now. massive frontier models on a "desktop rig". how awesome is this? and this is the worst it'll ever be.
0:18

i just tested to put GLM-5.2 on my rig. 753B parameter MoE. 2x RTX PRO 6000 Blackwells, Threadripper PRO 9995WX with 1TB DDR5. prefills at 64 tok/s. decode holds at 13-15. system RAM bandwidth is the bottleneck. running UD-Q4_K_XL 4-bit. 69.7% on Aider Polyglot, within two points of BF16. left the 5090 off. mismatched VRAM unbalances the split. this is what local AI looks like now. massive frontier models on a "desktop rig". how awesome is this? and this is the worst it'll ever be.

Samuel Cardillo

48,719 views • 1 month ago