antirez's banner

antirez

@antirez • 83,368 subscribers

Reproducible bugs are candies. I like programming too much for not liking automatic programming. In elenco AGCOM.

Shorts

Soon on GitHub. #C64 #retrocomputing

Soon on GitHub. #C64 #retrocomputing

31,566 Aufrufe

Iterating... Evals take time and are boring: but are a fundamental validation step of sane LLM inference. Let's try to make them as easy and fun to run as possible.

Iterating... Evals take time and are boring: but are a fundamental validation step of sane LLM inference. Let's try to make them as easy and fun to run as possible.

32,815 Aufrufe

DeepSeek v4 Flash has 43 blocks... so, even 4096 batches during the prefill can be smoothly visualized. Look at the new progress bar.

DeepSeek v4 Flash has 43 blocks... so, even 4096 batches during the prefill can be smoothly visualized. Look at the new progress bar.

18,422 Aufrufe

Sicily is the outside of Silo

Sicily is the outside of Silo

16,870 Aufrufe

First steps for a specialized DeepSeek v4 Flash inference engine focused on inference quality / stability at different quantizations, with networked API that is batching capable. This is the 2 bit quants model running on my M3 Max 128GB.

First steps for a specialized DeepSeek v4 Flash inference engine focused on inference quality / stability at different quantizations, with networked API that is batching capable. This is the 2 bit quants model running on my M3 Max 128GB.

14,220 Aufrufe

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

DwarfStar running DeepSeek v4.1 Flash on a 128GB M5 Max. I didn't expect with SSD streaming it could be so fast. Recent SSD streaming changes to retain the right experts surely helped, but also maybe DS4.1 uses the same experts more. Will push online when ready QA > ASAP.

DwarfStar running DeepSeek v4.1 Flash on a 128GB M5 Max. I didn't expect with SSD streaming it could be so fast. Recent SSD streaming changes to retain the right experts surely helped, but also maybe DS4.1 uses the same experts more. Will push online when ready QA > ASAP.

433,303 Aufrufe • vor 1 Monat

That's a bit slow. Streaming directly the K3 official hugging face 1.6TB of weights in mxfp4 in an m5 max 128gb.

That's a bit slow. Streaming directly the K3 official hugging face 1.6TB of weights in mxfp4 in an m5 max 128gb.

295,015 Aufrufe • vor 2 Monaten

So it was not impossible... just super hard. This is the 48k Spectrum...

So it was not impossible... just super hard. This is the 48k Spectrum...

83,139 Aufrufe • vor 28 Tagen

GLM 5.2, Q2_K routed experts (effectively ~2.6 bits) running with SSD streaming on an M5 Max 128GB computer.

GLM 5.2, Q2_K routed experts (effectively ~2.6 bits) running with SSD streaming on an M5 Max 128GB computer.

319,303 Aufrufe • vor 3 Monaten

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.

276,710 Aufrufe • vor 4 Monaten

If you need AI to do a search for you in the real world, ds4-agent is basically SOTA, because it can access the web sites without any limitations given that it uses your local Chrome browser (no, not in headless mode, that's the trick...), and DeepSeek v4 is great at search.

If you need AI to do a search for you in the real world, ds4-agent is basically SOTA, because it can access the web sites without any limitations given that it uses your local Chrome browser (no, not in headless mode, that's the trick...), and DeepSeek v4 is great at search.

150,417 Aufrufe • vor 4 Monaten

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.

173,075 Aufrufe • vor 5 Monaten

DeepSeek v4 Flash experimental with vision running (fast) on an m5 max computer and analyzing an image. The implementation for Metal / CUDA / ROCm is finished, just doing the last tests today before releasing it.

DeepSeek v4 Flash experimental with vision running (fast) on an m5 max computer and analyzing an image. The implementation for Metal / CUDA / ROCm is finished, just doing the last tests today before releasing it.

62,060 Aufrufe • vor 1 Monat

Crazy prefill speed of DwarfStar running DeepSeek v4 PRO in the DGX Station (private branch for now) using techniques that leverage the peculiar RAM + VRAM mixed setup of the Station. 3000 t/s but even more can be reached.

Crazy prefill speed of DwarfStar running DeepSeek v4 PRO in the DGX Station (private branch for now) using techniques that leverage the peculiar RAM + VRAM mixed setup of the Station. 3000 t/s but even more can be reached.

63,713 Aufrufe • vor 2 Monaten

The full Another World intro demo for the Spectrum 48k. Real VM implemented for the Z80 executed in real time. It works very well on the real ZX Spectrum I own. I'll release it soon, Z80 code included. CC: Eric Chahi

The full Another World intro demo for the Spectrum 48k. Real VM implemented for the Z80 executed in real time. It works very well on the real ZX Spectrum I own. I'll release it soon, Z80 code included. CC: Eric Chahi

23,123 Aufrufe • vor 27 Tagen

DGX Station: this is just the start of what it is possible to do (and why I claim DwarfStar could be *the* Station inference engine). DeepSeek v4 PRO Q2 with routed experts split among VRAM / RAM with kernels optimized for this peculiar setup. 45 t/s but can go faster.

DGX Station: this is just the start of what it is possible to do (and why I claim DwarfStar could be the Station inference engine). DeepSeek v4 PRO Q2 with routed experts split among VRAM / RAM with kernels optimized for this peculiar setup. 45 t/s but can go faster.

43,685 Aufrufe • vor 2 Monaten

First kinda working implementation of GLM 5.2 in DwarfStar. Will take some time to be good enough, but it is a promising start. 433 GB GGUF file. M3 Ultra 512GB.

First kinda working implementation of GLM 5.2 in DwarfStar. Will take some time to be good enough, but it is a promising start. 433 GB GGUF file. M3 Ultra 512GB.

69,587 Aufrufe • vor 4 Monaten

DS4 running on DGX Spark (GB10 / CUDA), private branch for now. 12 tokens/sec, the memory bandwidth is limited in this system, at 270GB/sec. But prefill is ways more alighed to M3 Max at ~200 t/s. I'll release when more mature, but it is almost sure that it will get merged.

DS4 running on DGX Spark (GB10 / CUDA), private branch for now. 12 tokens/sec, the memory bandwidth is limited in this system, at 270GB/sec. But prefill is ways more alighed to M3 Max at ~200 t/s. I'll release when more mature, but it is almost sure that it will get merged.

84,477 Aufrufe • vor 5 Monaten

It looks simple to use

It looks simple to use

20,287 Aufrufe • vor 1 Monat

GLM 5.2 flash Q2 (generated by myself with similar recipe of DS4 Flash of DwarfStar GGUF) running on an M5 Max with 128GB. The Q4 weights also work on tensor parallel inference in two M5 Max systems. Soon on GitHub. Still testing + CUDA + ROCm ...

GLM 5.2 flash Q2 (generated by myself with similar recipe of DS4 Flash of DwarfStar GGUF) running on an M5 Max with 128GB. The Q4 weights also work on tensor parallel inference in two M5 Max systems. Soon on GitHub. Still testing + CUDA + ROCm ...

29,541 Aufrufe • vor 1 Monat

I'm optimizing tensor parallel execution across two MacBook M5 Max systems using RDMA. This is how fast DeepSeek v4 Flash Exp Vision can run in MXFP4 (full precision) right now *without* DSpark, so this is the min speed you get in practice.

I'm optimizing tensor parallel execution across two MacBook M5 Max systems using RDMA. This is how fast DeepSeek v4 Flash Exp Vision can run in MXFP4 (full precision) right now without DSpark, so this is the min speed you get in practice.

27,006 Aufrufe • vor 1 Monat

Still didn't attempt any quality testing, but let's say that GPT 5.6 Sol was able almost unassisted to write this Laguna S2.1 inference implementation following the other two models in DwarfStar, and it is very fast, 50 t/s generation, very fast prefill as well.

Still didn't attempt any quality testing, but let's say that GPT 5.6 Sol was able almost unassisted to write this Laguna S2.1 inference implementation following the other two models in DwarfStar, and it is very fast, 50 t/s generation, very fast prefill as well.

43,712 Aufrufe • vor 3 Monaten

DeepSeek v4 PRO running in DwarfStar on a DGX Station. Can go faster, already peaks 50 t/s without DSpark: dynamic VRAM/RAM allocation of experts based on past statistics, 4k t/s prefill on big chunks. VRAM/RAM layer-major streaming. Mixed RAM/VRAM inference requires care.

DeepSeek v4 PRO running in DwarfStar on a DGX Station. Can go faster, already peaks 50 t/s without DSpark: dynamic VRAM/RAM allocation of experts based on past statistics, 4k t/s prefill on big chunks. VRAM/RAM layer-major streaming. Mixed RAM/VRAM inference requires care.

26,763 Aufrufe • vor 2 Monaten

DwarfStar with vision support for GLM 5.3 running on a MacBook m5max computer. The vision support is *not* quantized for max precision for now.

DwarfStar with vision support for GLM 5.3 running on a MacBook m5max computer. The vision support is not quantized for max precision for now.

22,664 Aufrufe • vor 1 Monat

Only 10 steps in this example to make the video short, yet H3 is quite fun.

Only 10 steps in this example to make the video short, yet H3 is quite fun.

26,809 Aufrufe • vor 2 Monaten