antirez

@antirez • 73,895 subscribers

Reproducible bugs are candies. I like programming too much for not liking automatic programming.

Shorts

32,727 views

16,870 views

18,422 views

14,176 views

Videos

sweetdream.ai

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Private Show

Join now for exclusive access

Free preview available • Premium content

317,088 views • 26 days ago

271,840 views • 1 month ago

148,707 views • 1 month ago

169,897 views • 2 months ago

69,419 views • 1 month ago

84,069 views • 2 months ago

39,786 views • 2 months ago

39,023 views • 2 months ago

44,464 views • 2 months ago

27,840 views • 2 months ago

24,805 views • 2 months ago

17,303 views • 1 month ago

25,147 views • 3 months ago

15,987 views • 5 months ago

No more content to load

Live Cam

antirez

Shorts

Iterating... Evals take time and are boring: but are a fundamental validation step of sane LLM inference. Let's try to make them as easy and fun to run as possible.

Sicily is the outside of Silo

DeepSeek v4 Flash has 43 blocks... so, even 4096 batches during the prefill can be smoothly visualized. Look at the new progress bar.

First steps for a specialized DeepSeek v4 Flash inference engine focused on inference quality / stability at different quantizations, with networked API that is batching capable. This is the 2 bit quants model running on my M3 Max 128GB.

Videos

Watch Anya Live

GLM 5.2, Q2_K routed experts (effectively ~2.6 bits) running with SSD streaming on an M5 Max 128GB computer.

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.

If you need AI to do a search for you in the real world, ds4-agent is basically SOTA, because it can access the web sites without any limitations given that it uses your local Chrome browser (no, not in headless mode, that's the trick...), and DeepSeek v4 is great at search.

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.

First kinda working implementation of GLM 5.2 in DwarfStar. Will take some time to be good enough, but it is a promising start. 433 GB GGUF file. M3 Ultra 512GB.

DS4 running on DGX Spark (GB10 / CUDA), private branch for now. 12 tokens/sec, the memory bandwidth is limited in this system, at 270GB/sec. But prefill is ways more alighed to M3 Max at ~200 t/s. I'll release when more mature, but it is almost sure that it will get merged.

For the DGX Spark owners. This is what you get with DS4 in your hardware. I want to post this to show how with fast prefill and not very fast generation, the system remains absolutely fine to use.

You didn't see this coming in ds4-agent, did you?

In case you have doubts about the q2 quants inference of DS4 (I noticed many don't trust the README claims), here is it analyzing Picol source code using the pi agent.

The first M5 max arrived! Many many thanks to our sponsors ⿻ Audrey Tang 唐鳳 and Niels Gron, the next will arrive on Monday.

Updating you on ds4-agent

Took the good work of the communtiy of DwarfStar and consolidating the Strix Halo support. It looks very good. More QA in the next days and the final merge soon.

Much better at 17 tokens/sec in M3 Max, fusing a few operations. 128gb m3 max.

Now we are talking. 2x audio length speed for the 1.7B Qwen3-ASR large (1.7b) transcription, on my very modest Hetzner 13th Gen i5 server. The pause you see initially is the encoding stage. -S 0 means: no segmentation, one offline pass.