Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

GLM 5.2, Q2_K routed experts (effectively ~2.6 bits) running with SSD streaming on an M5 Max 128GB computer.

antirez's profile picture

antirez

69,932 subscribers

127,814 görüntüleme • 2 gün önce •via X (Twitter)

Bilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.
4:15

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.

antirez

268,543 görüntüleme • 26 gün önce

~ 6.5 - 6.7 t/s for GLM 5.1 on M5 Max 128GB Added “Dense” model export, now model load is only 5s ! Experts are streaming from SSD, so we do not pre-load it. Added direct SSD->Slot memory path, removed prefetch... Many dead end experiments. See Export a “dense-only GGUF” and “Fast path ” in tools/flashmob-sidecar/README.md WIP branch for Flash-MoE-SSD
1:28

~ 6.5 - 6.7 t/s for GLM 5.1 on M5 Max 128GB Added “Dense” model export, now model load is only 5s ! Experts are streaming from SSD, so we do not pre-load it. Added direct SSD->Slot memory path, removed prefetch... Many dead end experiments. See Export a “dense-only GGUF” and “Fast path ” in tools/flashmob-sidecar/README.md WIP branch for Flash-MoE-SSD

Anemll

29,421 görüntüleme • 2 ay önce

Much better at 17 tokens/sec in M3 Max, fusing a few operations. 128gb m3 max.
0:49

Much better at 17 tokens/sec in M3 Max, fusing a few operations. 128gb m3 max.

antirez

25,121 görüntüleme • 2 ay önce

Now video on oMLX + Codex + qwen3.6-35b-a3b 6bit on an M5 Max 128GB. Video normal speed! Command used: onlx launch codex Local AI rocks 💪
0:56

Now video on oMLX + Codex + qwen3.6-35b-a3b 6bit on an M5 Max 128GB. Video normal speed! Command used: onlx launch codex Local AI rocks 💪

Ivan Fioravanti ᯅ

11,461 görüntüleme • 24 gün önce

Running GLM-4.7-Flash with OpenCode locally on M4 Max MacBook Pro. 4-bit model runs at 82 tok/sec. Prefill will get ~4x faster with M5 Max MacBook Pro (~28 Jan). EXO will also support disaggregating prefill and decode across devices, e.g. DGX Spark.
0:43

Running GLM-4.7-Flash with OpenCode locally on M4 Max MacBook Pro. 4-bit model runs at 82 tok/sec. Prefill will get ~4x faster with M5 Max MacBook Pro (~28 Jan). EXO will also support disaggregating prefill and decode across devices, e.g. DGX Spark.

Alex Cheema

127,902 görüntüleme • 5 ay önce

The first M5 max arrived! Many many thanks to our sponsors ⿻ Audrey Tang 唐鳳 and Niels Gron, the next will arrive on Monday.
0:42

The first M5 max arrived! Many many thanks to our sponsors ⿻ Audrey Tang 唐鳳 and Niels Gron, the next will arrive on Monday.

antirez

27,786 görüntüleme • 1 ay önce

First kinda working implementation of GLM 5.2 in DwarfStar. Will take some time to be good enough, but it is a promising start. 433 GB GGUF file. M3 Ultra 512GB.
0:36

First kinda working implementation of GLM 5.2 in DwarfStar. Will take some time to be good enough, but it is a promising start. 433 GB GGUF file. M3 Ultra 512GB.

antirez

69,010 görüntüleme • 9 gün önce

GLM 5.2 runs pretty fast on Modal.
0:24

GLM 5.2 runs pretty fast on Modal.

Charles 🎉 Frye

20,342 görüntüleme • 7 gün önce

WIP: First attempt to speed up prefill for Flash-MoE. Original repo did token-by-token without streamed experts. Added: Batched linear attention + batched full attention (Flash Attention style) with custom Metal kernels. Without experts: 6.2x faster prefill (11 -> 68 tok/s) With experts at full-attn layers only: 1.9x faster (11 -> 20.5 tok/s) — same output quality Qwen3.5-397B, 4-bit, 209GB, M5 Max 128GB 1/3
0:57

WIP: First attempt to speed up prefill for Flash-MoE. Original repo did token-by-token without streamed experts. Added: Batched linear attention + batched full attention (Flash Attention style) with custom Metal kernels. Without experts: 6.2x faster prefill (11 -> 68 tok/s) With experts at full-attn layers only: 1.9x faster (11 -> 20.5 tok/s) — same output quality Qwen3.5-397B, 4-bit, 209GB, M5 Max 128GB 1/3

Anemll

19,572 görüntüleme • 3 ay önce

Frontier-level agentic capabilities and long-horizon coding, open source and fully private. That's GLM 5.2, now on Venice. Here's a full walkthrough to build a retro video game with GLM 5.2 and OpenCode, starting from a single prompt:
7:54

Frontier-level agentic capabilities and long-horizon coding, open source and fully private. That's GLM 5.2, now on Venice. Here's a full walkthrough to build a retro video game with GLM 5.2 and OpenCode, starting from a single prompt:

Venice

26,885 görüntüleme • 13 gün önce

Video on LM Studio MLX v.1.8.5 + Codex + qwen3.6-35b-a3b 6bit on an M5 Max 128GB. Video is in normal speed to give you an idea of the experience. Command used: codex --oss -m qwen/qwen3.6-35b-a3b Local AI is becoming powerful 💪
1:18

Video on LM Studio MLX v.1.8.5 + Codex + qwen3.6-35b-a3b 6bit on an M5 Max 128GB. Video is in normal speed to give you an idea of the experience. Command used: codex --oss -m qwen/qwen3.6-35b-a3b Local AI is becoming powerful 💪

Ivan Fioravanti ᯅ

40,021 görüntüleme • 24 gün önce

we asked GLM 5.2 and Claude Opus 4.8 to find and fix the bugs in two different scripts while Opus 4.8 finished a few seconds faster in both tests, GLM 5.2 burned roughly 17k–38k fewer tokens and absolutely crushed it on cost, coming in 15x to 22x cheaper! Test 1: GLM 5.2 ($0.068) vs. Opus 4.8 ($1.06) Test 2: GLM 5.2 ($0.045) vs. Opus 4.8 ($1.02)
0:23

we asked GLM 5.2 and Claude Opus 4.8 to find and fix the bugs in two different scripts while Opus 4.8 finished a few seconds faster in both tests, GLM 5.2 burned roughly 17k–38k fewer tokens and absolutely crushed it on cost, coming in 15x to 22x cheaper! Test 1: GLM 5.2 ($0.068) vs. Opus 4.8 ($1.06) Test 2: GLM 5.2 ($0.045) vs. Opus 4.8 ($1.02)

GMI Cloud

85,530 görüntüleme • 4 gün önce

GLM-5.2 is a bigger deal than you think. > what GLM-5.2 actually scores (0:46) > "it's just a rich person's toy" (4:27) > why GLM-5.2 matters even if you never run it (7:19)
12:25

GLM-5.2 is a bigger deal than you think. > what GLM-5.2 actually scores (0:46) > "it's just a rich person's toy" (4:27) > why GLM-5.2 matters even if you never run it (7:19)

Rob Hallam

19,906 görüntüleme • 8 gün önce

GLM 5.2 vs Kimi K2.7 both were tested on same settings with same prompt > GLM one shotted everything in 35 mins > Kimi needed extra prompts to fix movements and took 30 mins super surpised by how good GLM 5.2 is, while being so cheap which one do you think won?
0:35

GLM 5.2 vs Kimi K2.7 both were tested on same settings with same prompt > GLM one shotted everything in 35 mins > Kimi needed extra prompts to fix movements and took 30 mins super surpised by how good GLM 5.2 is, while being so cheap which one do you think won?

J A Z I I

63,064 görüntüleme • 17 gün önce

Timelapse #156 (36 hrs) - Worked with the tiny corp on getting GLM 5.2 running on 8xMI300X (sglang won here) - Launched KernelBench-Mega and updated Kernelbench-Hard with h100 and b200 sweeps - Took care of boring business stuff - Did some training sweeps for specialized technical vocab audio model - Some bugs with putting kernelbench-mega and hard on cloud instances so had to do some reruns. Learned a lot though - Setting up my own local rl infra and profiling concurrency 128 rollouts with vllm. Became clear to me that I need to serve in nvfp4, use MoE only for throughput, reap so training doesn’t OOM, dig into the vllm kernel graph itself to not underutilize my hardware from poor flashinfer/cutlass selections for my rtx pro 6000 sm120 architecture - Might do online distillation from glm 5.2 but for now taking it one step at a time - Slept for a bit then woke up and showered - Fixed an issue with SGLang tensor parallel deadlock on GLM 5.2 architecture with MTP enabled - GLM 5.2 inference is 2-3x faster than coding plans and running on amd boxes - Spent time with family - Hung out with some friends - Recorded some yoctogpt lectures with the revamped notebook (high taste btw) - Setting up dflash training for GLM 5.2
0:30

Timelapse #156 (36 hrs) - Worked with the tiny corp on getting GLM 5.2 running on 8xMI300X (sglang won here) - Launched KernelBench-Mega and updated Kernelbench-Hard with h100 and b200 sweeps - Took care of boring business stuff - Did some training sweeps for specialized technical vocab audio model - Some bugs with putting kernelbench-mega and hard on cloud instances so had to do some reruns. Learned a lot though - Setting up my own local rl infra and profiling concurrency 128 rollouts with vllm. Became clear to me that I need to serve in nvfp4, use MoE only for throughput, reap so training doesn’t OOM, dig into the vllm kernel graph itself to not underutilize my hardware from poor flashinfer/cutlass selections for my rtx pro 6000 sm120 architecture - Might do online distillation from glm 5.2 but for now taking it one step at a time - Slept for a bit then woke up and showered - Fixed an issue with SGLang tensor parallel deadlock on GLM 5.2 architecture with MTP enabled - GLM 5.2 inference is 2-3x faster than coding plans and running on amd boxes - Spent time with family - Hung out with some friends - Recorded some yoctogpt lectures with the revamped notebook (high taste btw) - Setting up dflash training for GLM 5.2

Elliot Arledge

45,129 görüntüleme • 9 gün önce

GLM-5.2 delivers a substantial leap in app development capabilities, which also represent demanding long-horizon tasks. Results: - GLM-5.1: 21/70 - GLM-5.2: 48/70 - Claude Fable 5: 56/70 That's more than a twofold improvement from GLM-5.1 to GLM-5.2. These come from an internal benchmark of 35 challenging mobile development tasks, each run twice for a total of 70 trials. We measured task completion, defined as core features working without major issues.
0:45

GLM-5.2 delivers a substantial leap in app development capabilities, which also represent demanding long-horizon tasks. Results: - GLM-5.1: 21/70 - GLM-5.2: 48/70 - Claude Fable 5: 56/70 That's more than a twofold improvement from GLM-5.1 to GLM-5.2. These come from an internal benchmark of 35 challenging mobile development tasks, each run twice for a total of 70 trials. We measured task completion, defined as core features working without major issues.

Zixuan Li

299,340 görüntüleme • 11 gün önce

I was skeptical about giving GLM-5.2 the task to build a super mario snes style game in a single HTML file so i ran the test on Opus and also on GLM-5.2 Opus didnt even try to make a game or made something but it wasn't a game GLM-5.2 one shotted a pretty functional game.. WILD
0:45

I was skeptical about giving GLM-5.2 the task to build a super mario snes style game in a single HTML file so i ran the test on Opus and also on GLM-5.2 Opus didnt even try to make a game or made something but it wasn't a game GLM-5.2 one shotted a pretty functional game.. WILD

Hermes Agent Tips

32,805 görüntüleme • 11 gün önce

Llama 3.3 70B 4-bit running on a 128GB M4 Max with MLX LM (~12 toks/sec) with mactop on the right. "Good enough", local, private and open models are now more accessible than ever!
0:29

Llama 3.3 70B 4-bit running on a 128GB M4 Max with MLX LM (~12 toks/sec) with mactop on the right. "Good enough", local, private and open models are now more accessible than ever!

Ivan Fioravanti ᯅ

43,929 görüntüleme • 1 yıl önce

GLM-5.2确实有点东西
0:21

GLM-5.2确实有点东西

关木

58,140 görüntüleme • 17 gün önce