Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

I'm optimizing tensor parallel execution across two MacBook M5 Max systems using RDMA. This is how fast DeepSeek v4 Flash Exp Vision can run in MXFP4 (full precision) right now *without* DSpark, so this is the min speed you get in practice.

antirez's profile picture

antirez

80,319 subscribers

27,006 görüntüleme • 1 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

GLM 5.2 flash Q2 (generated by myself with similar recipe of DS4 Flash of DwarfStar GGUF) running on an M5 Max with 128GB. The Q4 weights also work on tensor parallel inference in two M5 Max systems. Soon on GitHub. Still testing + CUDA + ROCm ...
0:39

GLM 5.2 flash Q2 (generated by myself with similar recipe of DS4 Flash of DwarfStar GGUF) running on an M5 Max with 128GB. The Q4 weights also work on tensor parallel inference in two M5 Max systems. Soon on GitHub. Still testing + CUDA + ROCm ...

antirez

29,541 görüntüleme • 1 ay önce

DeepSeek v4 Flash experimental with vision running (fast) on an m5 max computer and analyzing an image. The implementation for Metal / CUDA / ROCm is finished, just doing the last tests today before releasing it.
1:19

DeepSeek v4 Flash experimental with vision running (fast) on an m5 max computer and analyzing an image. The implementation for Metal / CUDA / ROCm is finished, just doing the last tests today before releasing it.

antirez

62,060 görüntüleme • 1 ay önce

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.
4:15

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.

antirez

276,710 görüntüleme • 4 ay önce

DwarfStar with vision support for GLM 5.3 running on a MacBook m5max computer. The vision support is *not* quantized for max precision for now.
0:51

DwarfStar with vision support for GLM 5.3 running on a MacBook m5max computer. The vision support is *not* quantized for max precision for now.

antirez

22,664 görüntüleme • 1 ay önce

DeepSeek V4 Flash Vision Exp is up & running on 2x DGX Sparks Native vision!!! 🔥🔥
0:23

DeepSeek V4 Flash Vision Exp is up & running on 2x DGX Sparks Native vision!!! 🔥🔥

Mia

29,326 görüntüleme • 1 ay önce

Initial DeepSeek v4 and v4 Flash support added to anemll-flash-llama.cpp! 🚀 M5 MAX 128GB can run full DeepSeek-V4 with 1.6T params in original FP8/FP4 weights from SSD without requantization! One-click scripts to convert HF safetensors to dense + MoE sidecar (no GGUF needed) Inference and server examples Prefill and decoding benchmarks Branch and docs here: This is WIP and I plan to add cleanup and more testing this week. Feedback welcome!
0:21

Initial DeepSeek v4 and v4 Flash support added to anemll-flash-llama.cpp! 🚀 M5 MAX 128GB can run full DeepSeek-V4 with 1.6T params in original FP8/FP4 weights from SSD without requantization! One-click scripts to convert HF safetensors to dense + MoE sidecar (no GGUF needed) Inference and server examples Prefill and decoding benchmarks Branch and docs here: This is WIP and I plan to add cleanup and more testing this week. Feedback welcome!

Anemll

32,250 görüntüleme • 5 ay önce

I just saw AGI, what the heck. Deepseek-v4-flash-0731 invented vision for itself, this is so incredible. Result in the next post:
4:11

I just saw AGI, what the heck. Deepseek-v4-flash-0731 invented vision for itself, this is so incredible. Result in the next post:

0xSero

175,626 görüntüleme • 2 ay önce

That's a bit slow. Streaming directly the K3 official hugging face 1.6TB of weights in mxfp4 in an m5 max 128gb.
1:06

That's a bit slow. Streaming directly the K3 official hugging face 1.6TB of weights in mxfp4 in an m5 max 128gb.

antirez

295,015 görüntüleme • 2 ay önce

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.
1:24

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.

antirez

173,075 görüntüleme • 5 ay önce

We've got Opus 4.8 at home! Running DeepSeek V4 Flash Vision-Exp on 2 x DGX Spark with OMP. This is a mini-benchmark firmware analysis task I've been using to test out different models/harnesses. Opus 4.8 didn't find anything in this firmware that DeepSeek didn't. Thanks Mia
0:24

We've got Opus 4.8 at home! Running DeepSeek V4 Flash Vision-Exp on 2 x DGX Spark with OMP. This is a mini-benchmark firmware analysis task I've been using to test out different models/harnesses. Opus 4.8 didn't find anything in this firmware that DeepSeek didn't. Thanks Mia

Matt Brown

25,804 görüntüleme • 1 ay önce

Crazy prefill speed of DwarfStar running DeepSeek v4 PRO in the DGX Station (private branch for now) using techniques that leverage the peculiar RAM + VRAM mixed setup of the Station. 3000 t/s but even more can be reached.
0:23

Crazy prefill speed of DwarfStar running DeepSeek v4 PRO in the DGX Station (private branch for now) using techniques that leverage the peculiar RAM + VRAM mixed setup of the Station. 3000 t/s but even more can be reached.

antirez

63,713 görüntüleme • 2 ay önce

I'm now used to this speed 🤣 Running DeepSeek-V4-Flash-IQ2XXS on my single 3090 and 64GB RAM tg is fine, its the prefill that actually hurts.
1:04

I'm now used to this speed 🤣 Running DeepSeek-V4-Flash-IQ2XXS on my single 3090 and 64GB RAM tg is fine, its the prefill that actually hurts.

AJ

20,999 görüntüleme • 3 ay önce

SGLang now supports DSpark, enabling confidence-driven, variable-length verification for speculative decoding 🎉 DSpark addresses a key bottleneck under load: instead of verifying every draft token, it verifies only where the draft model is confident, so the gains hold even as batch size scales. We heavily optimized variable-length verification in SGLang. Across batch sizes 1 to 256, DSpark gives the best throughput/latency tradeoff on DeepSeek-V4-Flash, ahead of both MTP and non-spec. At high concurrency, dynamic scheduling provides up to ~20% higher throughput compared to a fixed budget, while maintaining high verification quality across workloads. With fused kernels and zero-overhead scheduling, DeepSeek-V4-Pro reaches 383.7 tok/s at B=1 on B300. DSpark is now available in SGLang with support for Qwen3 and DeepSeek-V4. Thanks DeepSeek for open-sourcing! Blog with full technical details and commands to run below 👇
0:33

SGLang now supports DSpark, enabling confidence-driven, variable-length verification for speculative decoding 🎉 DSpark addresses a key bottleneck under load: instead of verifying every draft token, it verifies only where the draft model is confident, so the gains hold even as batch size scales. We heavily optimized variable-length verification in SGLang. Across batch sizes 1 to 256, DSpark gives the best throughput/latency tradeoff on DeepSeek-V4-Flash, ahead of both MTP and non-spec. At high concurrency, dynamic scheduling provides up to ~20% higher throughput compared to a fixed budget, while maintaining high verification quality across workloads. With fused kernels and zero-overhead scheduling, DeepSeek-V4-Pro reaches 383.7 tok/s at B=1 on B300. DSpark is now available in SGLang with support for Qwen3 and DeepSeek-V4. Thanks DeepSeek for open-sourcing! Blog with full technical details and commands to run below 👇

LMSYS Org

170,752 görüntüleme • 3 ay önce

DwarfStar running DeepSeek v4.1 Flash on a 128GB M5 Max. I didn't expect with SSD streaming it could be so fast. Recent SSD streaming changes to retain the right experts surely helped, but also maybe DS4.1 uses the same experts more. Will push online when ready QA > ASAP.
2:18

DwarfStar running DeepSeek v4.1 Flash on a 128GB M5 Max. I didn't expect with SSD streaming it could be so fast. Recent SSD streaming changes to retain the right experts surely helped, but also maybe DS4.1 uses the same experts more. Will push online when ready QA > ASAP.

antirez

433,303 görüntüleme • 1 ay önce

DeepSeek V4 Flash 0731 in Hermes Agent and one prompt, took 32 minutes and cost 0.07$, this model is so cheap to the point where 2 dollars can last you a full day.
1:03

DeepSeek V4 Flash 0731 in Hermes Agent and one prompt, took 32 minutes and cost 0.07$, this model is so cheap to the point where 2 dollars can last you a full day.

Elshayib

689,761 görüntüleme • 2 ay önce

$0.04. That's what DeepSeek-V4-Flash-Vision-Exp cost to watch the Grok Bot Grok Bot library video. DeepSeek V4 Flash ran the same prompt through ego lite, blind to every frame: 27.9 minutes, $0.14. 3.3x the cost, for a model spending the whole run trying to see.
0:31

$0.04. That's what DeepSeek-V4-Flash-Vision-Exp cost to watch the Grok Bot Grok Bot library video. DeepSeek V4 Flash ran the same prompt through ego lite, blind to every frame: 27.9 minutes, $0.14. 3.3x the cost, for a model spending the whole run trying to see.

ego

24,785 görüntüleme • 2 ay önce

If you need AI to do a search for you in the real world, ds4-agent is basically SOTA, because it can access the web sites without any limitations given that it uses your local Chrome browser (no, not in headless mode, that's the trick...), and DeepSeek v4 is great at search.
1:29

If you need AI to do a search for you in the real world, ds4-agent is basically SOTA, because it can access the web sites without any limitations given that it uses your local Chrome browser (no, not in headless mode, that's the trick...), and DeepSeek v4 is great at search.

antirez

150,417 görüntüleme • 4 ay önce

DeepSeek v4 PRO running in DwarfStar on a DGX Station. Can go faster, already peaks 50 t/s without DSpark: dynamic VRAM/RAM allocation of experts based on past statistics, 4k t/s prefill on big chunks. VRAM/RAM layer-major streaming. Mixed RAM/VRAM inference requires care.
0:27

DeepSeek v4 PRO running in DwarfStar on a DGX Station. Can go faster, already peaks 50 t/s without DSpark: dynamic VRAM/RAM allocation of experts based on past statistics, 4k t/s prefill on big chunks. VRAM/RAM layer-major streaming. Mixed RAM/VRAM inference requires care.

antirez

26,763 görüntüleme • 2 ay önce

Deepseek V4.1 Flash Running LOCALLY in The Deepseek Harness. This is my FAVORITE Harness right now. Your OWN Local CODEX.
1:47

Deepseek V4.1 Flash Running LOCALLY in The Deepseek Harness. This is my FAVORITE Harness right now. Your OWN Local CODEX.

Tech2Wild

18,635 görüntüleme • 1 ay önce

DeepSeek V4.1 Flash just dropped, and holy shit this thing is fast,I’m getting almost 400 t/s in my video tests. The speed is honestly crazy, I’m running my own benchmark right now and I’ll share all the results once I’m done. You just need to go to the DeepSeek harness and use deepseek-v4.1-flash-expires-on-0910. Speed wise, it’s actually competing with Muse Spark 1.3!
1:05

DeepSeek V4.1 Flash just dropped, and holy shit this thing is fast,I’m getting almost 400 t/s in my video tests. The speed is honestly crazy, I’m running my own benchmark right now and I’ll share all the results once I’m done. You just need to go to the DeepSeek harness and use deepseek-v4.1-flash-expires-on-0910. Speed wise, it’s actually competing with Muse Spark 1.3!

Iam_

102,870 görüntüleme • 1 ay önce