Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

First kinda working implementation of GLM 5.2 in DwarfStar. Will take some time to be good enough, but it is a promising start. 433 GB GGUF file. M3 Ultra 512GB.

antirez's profile picture

antirez

80,323 subscribers

69,587 Aufrufe • vor 3 Monaten •via X (Twitter)

GamingBildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.
1:24

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.

antirez

172,816 Aufrufe • vor 4 Monaten

GLM 5.2 flash Q2 (generated by myself with similar recipe of DS4 Flash of DwarfStar GGUF) running on an M5 Max with 128GB. The Q4 weights also work on tensor parallel inference in two M5 Max systems. Soon on GitHub. Still testing + CUDA + ROCm ...
0:39

GLM 5.2 flash Q2 (generated by myself with similar recipe of DS4 Flash of DwarfStar GGUF) running on an M5 Max with 128GB. The Q4 weights also work on tensor parallel inference in two M5 Max systems. Soon on GitHub. Still testing + CUDA + ROCm ...

antirez

29,432 Aufrufe • vor 1 Monat

I did it! It works! Using GLM-4.7-4bit with mlx_lm.server and opencode to fix real code locally! 🔥 Here single M3 Ultra 512GB, nex step phase will be 2 using Tensor Parallelism and then apply same changes to exo. Prefill is slow on a single machine, but generation is good.
1:44

I did it! It works! Using GLM-4.7-4bit with mlx_lm.server and opencode to fix real code locally! 🔥 Here single M3 Ultra 512GB, nex step phase will be 2 using Tensor Parallelism and then apply same changes to exo. Prefill is slow on a single machine, but generation is good.

Ivan Fioravanti ᯅ

44,000 Aufrufe • vor 9 Monaten

GLM-5.2 8bit running on two M3 Ultra 512GB with MLX distributed? Here it is! 🚀 Decode speed: 17.9 tokens/sec 🔥 Memory used: ~ 760GB 👀 Again keep in mind it's a preliminary PR by super Pedro Cuenca still a WIP!
0:24

GLM-5.2 8bit running on two M3 Ultra 512GB with MLX distributed? Here it is! 🚀 Decode speed: 17.9 tokens/sec 🔥 Memory used: ~ 760GB 👀 Again keep in mind it's a preliminary PR by super Pedro Cuenca still a WIP!

Ivan Fioravanti

34,219 Aufrufe • vor 3 Monaten

MLX GLM 5.2 Distributed on two M3 Ultra 512GB 🔥 One M3 Ultra: 18.8 tokens/sec Two M3 Ultra: 23.4 tokens/sec Context: - PR by Pedro Cuenca is still open and probably there is room for improvement: - basic generation test to measure decoding performance here, I will do a full context benchmarking once PR is more mature - nvfp4 quantization used - Video alternates standard speed and x20, with one Mac first and distributed later. Enjoy! 🙌🏻
0:35

MLX GLM 5.2 Distributed on two M3 Ultra 512GB 🔥 One M3 Ultra: 18.8 tokens/sec Two M3 Ultra: 23.4 tokens/sec Context: - PR by Pedro Cuenca is still open and probably there is room for improvement: - basic generation test to measure decoding performance here, I will do a full context benchmarking once PR is more mature - nvfp4 quantization used - Video alternates standard speed and x20, with one Mac first and distributed later. Enjoy! 🙌🏻

Ivan Fioravanti ᯅ

88,821 Aufrufe • vor 3 Monaten

DwarfStar with vision support for GLM 5.3 running on a MacBook m5max computer. The vision support is *not* quantized for max precision for now.
0:51

DwarfStar with vision support for GLM 5.3 running on a MacBook m5max computer. The vision support is *not* quantized for max precision for now.

antirez

22,664 Aufrufe • vor 1 Monat

DGX Station: this is just the start of what it is possible to do (and why I claim DwarfStar could be *the* Station inference engine). DeepSeek v4 PRO Q2 with routed experts split among VRAM / RAM with kernels optimized for this peculiar setup. 45 t/s but can go faster.
0:53

DGX Station: this is just the start of what it is possible to do (and why I claim DwarfStar could be *the* Station inference engine). DeepSeek v4 PRO Q2 with routed experts split among VRAM / RAM with kernels optimized for this peculiar setup. 45 t/s but can go faster.

antirez

41,545 Aufrufe • vor 1 Monat

Still didn't attempt any quality testing, but let's say that GPT 5.6 Sol was able almost unassisted to write this Laguna S2.1 inference implementation following the other two models in DwarfStar, and it is very fast, 50 t/s generation, very fast prefill as well.
0:39

Still didn't attempt any quality testing, but let's say that GPT 5.6 Sol was able almost unassisted to write this Laguna S2.1 inference implementation following the other two models in DwarfStar, and it is very fast, 50 t/s generation, very fast prefill as well.

antirez

43,676 Aufrufe • vor 2 Monaten

Took the good work of the communtiy of DwarfStar and consolidating the Strix Halo support. It looks very good. More QA in the next days and the final merge soon.
1:20

Took the good work of the communtiy of DwarfStar and consolidating the Strix Halo support. It looks very good. More QA in the next days and the final merge soon.

antirez

17,567 Aufrufe • vor 4 Monaten

GLM-5 runs with mlx-lm on a single 512GB M3 Ultra in Q4. It's quite good in my initial testing and pretty fast as well. It generated a highly functional space invaders game using 7.1k tokens at 15.4 tok/s and 419GB memory. Thanks to Gökdeniz Gülmez and Tarjei Mandt for the port.
1:11

GLM-5 runs with mlx-lm on a single 512GB M3 Ultra in Q4. It's quite good in my initial testing and pretty fast as well. It generated a highly functional space invaders game using 7.1k tokens at 15.4 tok/s and 419GB memory. Thanks to Gökdeniz Gülmez and Tarjei Mandt for the port.

Awni Hannun

60,694 Aufrufe • vor 7 Monaten

All of the election fraud evidence I have obtained will be posted here within this thread. It will take some time to get to all of it, but I will eventually. The first one is this, a video of the switching of votes, how it works... and an example of it occuring in PA in 2020.
2:45

All of the election fraud evidence I have obtained will be posted here within this thread. It will take some time to get to all of it, but I will eventually. The first one is this, a video of the switching of votes, how it works... and an example of it occuring in PA in 2020.

NotKrassenstein

175,186 Aufrufe • vor 1 Jahr

GLM 5.2, Q2_K routed experts (effectively ~2.6 bits) running with SSD streaming on an M5 Max 128GB computer.
1:31

GLM 5.2, Q2_K routed experts (effectively ~2.6 bits) running with SSD streaming on an M5 Max 128GB computer.

antirez

319,142 Aufrufe • vor 3 Monaten

DwarfStar running DeepSeek v4.1 Flash on a 128GB M5 Max. I didn't expect with SSD streaming it could be so fast. Recent SSD streaming changes to retain the right experts surely helped, but also maybe DS4.1 uses the same experts more. Will push online when ready QA > ASAP.
2:18

DwarfStar running DeepSeek v4.1 Flash on a 128GB M5 Max. I didn't expect with SSD streaming it could be so fast. Recent SSD streaming changes to retain the right experts surely helped, but also maybe DS4.1 uses the same experts more. Will push online when ready QA > ASAP.

antirez

432,009 Aufrufe • vor 26 Tagen

MPJ: "Some of the shots and some of the ways they (the Nuggets) probably could have used me we had enough good players to where we could create a good shot in a different way...it is good to be able to kinda show what I'm capable of"
1:30

MPJ: "Some of the shots and some of the ways they (the Nuggets) probably could have used me we had enough good players to where we could create a good shot in a different way...it is good to be able to kinda show what I'm capable of"

Oh No He Didn't

158,326 Aufrufe • vor 9 Monaten

DS4 running on DGX Spark (GB10 / CUDA), private branch for now. 12 tokens/sec, the memory bandwidth is limited in this system, at 270GB/sec. But prefill is ways more alighed to M3 Max at ~200 t/s. I'll release when more mature, but it is almost sure that it will get merged.
1:34

DS4 running on DGX Spark (GB10 / CUDA), private branch for now. 12 tokens/sec, the memory bandwidth is limited in this system, at 270GB/sec. But prefill is ways more alighed to M3 Max at ~200 t/s. I'll release when more mature, but it is almost sure that it will get merged.

antirez

84,127 Aufrufe • vor 5 Monaten

Crazy prefill speed of DwarfStar running DeepSeek v4 PRO in the DGX Station (private branch for now) using techniques that leverage the peculiar RAM + VRAM mixed setup of the Station. 3000 t/s but even more can be reached.
0:23

Crazy prefill speed of DwarfStar running DeepSeek v4 PRO in the DGX Station (private branch for now) using techniques that leverage the peculiar RAM + VRAM mixed setup of the Station. 3000 t/s but even more can be reached.

antirez

63,235 Aufrufe • vor 1 Monat

Conservative prosecutors are resigning instead of carrying out Trump DOJ political orders. “You will eventually find someone who is enough of a fool, or enough of a coward, to file your motion. But it was never going to be me.” Republicans in Congress—STAND UP.
1:42

Conservative prosecutors are resigning instead of carrying out Trump DOJ political orders. “You will eventually find someone who is enough of a fool, or enough of a coward, to file your motion. But it was never going to be me.” Republicans in Congress—STAND UP.

Amy Klobuchar

385,534 Aufrufe • vor 1 Jahr

Zavier Gozo is the best attacking talent I’ve seen out of MLS in quite some time. Believe the hype; he’s good enough to start in a Champions League team.
1:01

Zavier Gozo is the best attacking talent I’ve seen out of MLS in quite some time. Believe the hype; he’s good enough to start in a Champions League team.

Zach Lowy

86,496 Aufrufe • vor 5 Monaten

I don't have anywhere to go for Halloween, but I kinda want to start working on a mermaid costume anyway. I figured it'd at least be fun to take some pictures in...plus I already have the accessories and hair. Lol 🧜‍♀️🦄🩷
0:24

I don't have anywhere to go for Halloween, but I kinda want to start working on a mermaid costume anyway. I figured it'd at least be fun to take some pictures in...plus I already have the accessories and hair. Lol 🧜‍♀️🦄🩷

Mindy MF Robinson 🦄

13,362 Aufrufe • vor 8 Tagen