Loading video...

Video Failed to Load

Go Home

Initial DeepSeek v4 and v4 Flash support added to anemll-flash-llama.cpp! 🚀 M5 MAX 128GB can run full DeepSeek-V4 with 1.6T params in original FP8/FP4 weights from SSD without requantization! One-click scripts to convert HF safetensors to dense + MoE sidecar (no GGUF needed) Inference and server examples Prefill and...

anemll's profile picture

Anemll

4,383 subscribers

32,250 views • 6 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

~ 6.5 - 6.7 t/s for GLM 5.1 on M5 Max 128GB Added “Dense” model export, now model load is only 5s ! Experts are streaming from SSD, so we do not pre-load it. Added direct SSD->Slot memory path, removed prefetch... Many dead end experiments. See Export a “dense-only GGUF” and “Fast path ” in tools/flashmob-sidecar/README.md WIP branch for Flash-MoE-SSD
1:28

~ 6.5 - 6.7 t/s for GLM 5.1 on M5 Max 128GB Added “Dense” model export, now model load is only 5s ! Experts are streaming from SSD, so we do not pre-load it. Added direct SSD->Slot memory path, removed prefetch... Many dead end experiments. See Export a “dense-only GGUF” and “Fast path ” in tools/flashmob-sidecar/README.md WIP branch for Flash-MoE-SSD

Anemll

29,421 views • 6 months ago

Added web search to Papers with Code's upcoming chat interface This runs on Hugging Face Inference Providers via Baseten (new DeepSeek-V4 Flash), using Exa's MCP server, running inside a Modal container Drop your HF username, and I'll add you, eager for feedback :)
0:24

Added web search to Papers with Code's upcoming chat interface This runs on Hugging Face Inference Providers via Baseten (new DeepSeek-V4 Flash), using Exa's MCP server, running inside a Modal container Drop your HF username, and I'll add you, eager for feedback :)

Niels Rogge

20,030 views • 2 months ago

.DeepSeek-V4-Pro-0813 is live on DigitalOcean Serverless Inference and Inference Router. 🚀 1.6T params / 49B active, 1M-token context, AA Intelligence Index of 53 at max reasoning effort.
0:37

.DeepSeek-V4-Pro-0813 is live on DigitalOcean Serverless Inference and Inference Router. 🚀 1.6T params / 49B active, 1M-token context, AA Intelligence Index of 53 at max reasoning effort.

DigitalOcean

347,944 views • 2 months ago

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.
4:15

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.

antirez

276,931 views • 4 months ago

WIP: First attempt to speed up prefill for Flash-MoE. Original repo did token-by-token without streamed experts. Added: Batched linear attention + batched full attention (Flash Attention style) with custom Metal kernels. Without experts: 6.2x faster prefill (11 -> 68 tok/s) With experts at full-attn layers only: 1.9x faster (11 -> 20.5 tok/s) — same output quality Qwen3.5-397B, 4-bit, 209GB, M5 Max 128GB 1/3
0:57

WIP: First attempt to speed up prefill for Flash-MoE. Original repo did token-by-token without streamed experts. Added: Batched linear attention + batched full attention (Flash Attention style) with custom Metal kernels. Without experts: 6.2x faster prefill (11 -> 68 tok/s) With experts at full-attn layers only: 1.9x faster (11 -> 20.5 tok/s) — same output quality Qwen3.5-397B, 4-bit, 209GB, M5 Max 128GB 1/3

Anemll

19,572 views • 7 months ago

I'm now used to this speed 🤣 Running DeepSeek-V4-Flash-IQ2XXS on my single 3090 and 64GB RAM tg is fine, its the prefill that actually hurts.
1:04

I'm now used to this speed 🤣 Running DeepSeek-V4-Flash-IQ2XXS on my single 3090 and 64GB RAM tg is fine, its the prefill that actually hurts.

AJ

20,999 views • 4 months ago

DeepSeek V4 Flash 0731 in Hermes Agent and one prompt, took 32 minutes and cost 0.07$, this model is so cheap to the point where 2 dollars can last you a full day.
1:03

DeepSeek V4 Flash 0731 in Hermes Agent and one prompt, took 32 minutes and cost 0.07$, this model is so cheap to the point where 2 dollars can last you a full day.

Elshayib

689,761 views • 3 months ago

Timelapse #177 (14 hrs) - Picked up an RTX Pro 6000 Blackwell and wired in another 1000W PSU - running two of them now - DeepSeek v4 Flash running locally on both GPUs - Distilling DeepSeek v4 Flash down into Nanbeige (still cooking) - Kernel bench runs for DeepSeek v4 Flash and Qwen 3.8 Max - Got it into chat but couldn't reliably get it into the OMP harness yet - Private contract work - days like this are stuff I actually want to build. and when it's the high-intensity grind, i still have to show up and get it done
0:30

Timelapse #177 (14 hrs) - Picked up an RTX Pro 6000 Blackwell and wired in another 1000W PSU - running two of them now - DeepSeek v4 Flash running locally on both GPUs - Distilling DeepSeek v4 Flash down into Nanbeige (still cooking) - Kernel bench runs for DeepSeek v4 Flash and Qwen 3.8 Max - Got it into chat but couldn't reliably get it into the OMP harness yet - Private contract work - days like this are stuff I actually want to build. and when it's the high-intensity grind, i still have to show up and get it done

Elliot Arledge

17,401 views • 2 months ago

DeepSeek-V4-Flash-0731 is out and it's amazing. Run it locally today on NVIDIA hardware.
0:58

DeepSeek-V4-Flash-0731 is out and it's amazing. Run it locally today on NVIDIA hardware.

NVIDIA RTX Spark

15,979 views • 2 months ago

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.
1:24

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.

antirez

173,075 views • 5 months ago

I can't believe this works, but I got DeepSeek-V4-Flash (284B params) running on a Raspberry Pi 5 (8GB edition) at >1tok/s @ ~8W during full-tilt inference! It uses an untouched copy of antirez's GGUF. Took 160+ experiments over 5 days between GPT-5.5 xhigh and Opus 4.8 max.
0:32

I can't believe this works, but I got DeepSeek-V4-Flash (284B params) running on a Raspberry Pi 5 (8GB edition) at >1tok/s @ ~8W during full-tilt inference! It uses an untouched copy of antirez's GGUF. Took 160+ experiments over 5 days between GPT-5.5 xhigh and Opus 4.8 max.

Dan Woods

185,556 views • 5 months ago

DeepSeek V4 Flash & V4 Pro Official API Speed Test V4 Flash: 80.63 tok/s V4 Pro: 36.72 tok/s
0:22

DeepSeek V4 Flash & V4 Pro Official API Speed Test V4 Flash: 80.63 tok/s V4 Pro: 36.72 tok/s

stevibe

17,555 views • 6 months ago

DeepSeek v4 Flash in one DGX Spark in action
0:30

DeepSeek v4 Flash in one DGX Spark in action

Mia

23,326 views • 2 months ago

I'm optimizing tensor parallel execution across two MacBook M5 Max systems using RDMA. This is how fast DeepSeek v4 Flash Exp Vision can run in MXFP4 (full precision) right now *without* DSpark, so this is the min speed you get in practice.
1:29

I'm optimizing tensor parallel execution across two MacBook M5 Max systems using RDMA. This is how fast DeepSeek v4 Flash Exp Vision can run in MXFP4 (full precision) right now *without* DSpark, so this is the min speed you get in practice.

antirez

27,006 views • 1 month ago

For anyone wandering what does it mean to run ds4-agent locally on an M5 Max using DeepSeek V4 Flash q2-imatrix gguf model. Here's a video of ds4 updating itself, adding a way to leverage HF_HOME for gguf models. Future of Local AI is bright!
2:31

For anyone wandering what does it mean to run ds4-agent locally on an M5 Max using DeepSeek V4 Flash q2-imatrix gguf model. Here's a video of ds4 updating itself, adding a way to leverage HF_HOME for gguf models. Future of Local AI is bright!

Ivan Fioravanti ᯅ

30,452 views • 5 months ago

My first long form video on @x! I'm testing DeepSeek V4 Flash inside of Codex and doing a pull request with it to see if it's any good.
19:09

My first long form video on @x! I'm testing DeepSeek V4 Flash inside of Codex and doing a pull request with it to see if it's any good.

Gui Bibeau e/acc

14,343 views • 2 months ago

FREE DeepSeek V4 Pro and V4 Flash API. Runtime by Bad Theory Labs just dropped it. Same platform, new free model, same 2-minute setup. > Runtime already gave you free Claude Opus 4.8 and GPT 5.5 access. Now they added DeepSeek V4 Pro and V4 Flash to the free tier. Absolutely FREE, YES. Same exact flow as before: → Go to runtime(.)badtheorylabs(.)com → Sign up with Google → Fill in your details during onboarding → Free balance lands automatically(But you don't need it to use DeepSeek - it's completely free right now) → Grab your API key from the dashboard → Plug into OpenCode, OpenClaw, or Hermes - done DeepSeek V4 Pro is one of the strongest coding and reasoning models available right now. Free tier also includes btl-2 - auto-routes between GLM, OpenAI, Anthropic, and Gemma. 10 million tokens per month at zero cost. 344 models total. One OpenAI-compatible endpoint. No code rebuild, no card, no limits hunting. >> Just start using the API, and when choosing a provider, select DeepSeek I tested it myself - fully working. Bookmark this before the free tier fills up.
0:23

FREE DeepSeek V4 Pro and V4 Flash API. Runtime by Bad Theory Labs just dropped it. Same platform, new free model, same 2-minute setup. > Runtime already gave you free Claude Opus 4.8 and GPT 5.5 access. Now they added DeepSeek V4 Pro and V4 Flash to the free tier. Absolutely FREE, YES. Same exact flow as before: → Go to runtime(.)badtheorylabs(.)com → Sign up with Google → Fill in your details during onboarding → Free balance lands automatically(But you don't need it to use DeepSeek - it's completely free right now) → Grab your API key from the dashboard → Plug into OpenCode, OpenClaw, or Hermes - done DeepSeek V4 Pro is one of the strongest coding and reasoning models available right now. Free tier also includes btl-2 - auto-routes between GLM, OpenAI, Anthropic, and Gemma. 10 million tokens per month at zero cost. 344 models total. One OpenAI-compatible endpoint. No code rebuild, no card, no limits hunting. >> Just start using the API, and when choosing a provider, select DeepSeek I tested it myself - fully working. Bookmark this before the free tier fills up.

Atenov int.

27,594 views • 4 months ago

This is probably my most important post about DeepSeek V4 Flash. Do NOT judge this model after testing only one harness. Pi, Claude Code, and OpenCode produced dramatically different results—and OpenCode completely changed my verdict. You really need to read the details below.
0:24

This is probably my most important post about DeepSeek V4 Flash. Do NOT judge this model after testing only one harness. Pi, Claude Code, and OpenCode produced dramatically different results—and OpenCode completely changed my verdict. You really need to read the details below.

OmedTheVibeCoder

68,806 views • 2 months ago

DeepSeek v4 Flash experimental with vision running (fast) on an m5 max computer and analyzing an image. The implementation for Metal / CUDA / ROCm is finished, just doing the last tests today before releasing it.
1:19

DeepSeek v4 Flash experimental with vision running (fast) on an m5 max computer and analyzing an image. The implementation for Metal / CUDA / ROCm is finished, just doing the last tests today before releasing it.

antirez

62,060 views • 1 month ago

DeepSeek-V4-Flash-Spark and Spark-Mini were born today. 1 command setup to Pi / vllm-studio / docker deployment. All this is running local on a single DGX Spark.
3:57

DeepSeek-V4-Flash-Spark and Spark-Mini were born today. 1 command setup to Pi / vllm-studio / docker deployment. All this is running local on a single DGX Spark.

0xSero

20,019 views • 5 months ago