Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Initial DeepSeek v4 and v4 Flash support added to anemll-flash-llama.cpp! 🚀 M5 MAX 128GB can run full DeepSeek-V4 with 1.6T params in original FP8/FP4 weights from SSD without requantization! One-click scripts to convert HF safetensors to dense + MoE sidecar (no GGUF needed) Inference and server examples Prefill and...

anemll's profile picture

Anemll

4,241 subscribers

32,085 görüntüleme • 2 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

~ 6.5 - 6.7 t/s for GLM 5.1 on M5 Max 128GB Added “Dense” model export, now model load is only 5s ! Experts are streaming from SSD, so we do not pre-load it. Added direct SSD->Slot memory path, removed prefetch... Many dead end experiments. See Export a “dense-only GGUF” and “Fast path ” in tools/flashmob-sidecar/README.md WIP branch for Flash-MoE-SSD
1:28

~ 6.5 - 6.7 t/s for GLM 5.1 on M5 Max 128GB Added “Dense” model export, now model load is only 5s ! Experts are streaming from SSD, so we do not pre-load it. Added direct SSD->Slot memory path, removed prefetch... Many dead end experiments. See Export a “dense-only GGUF” and “Fast path ” in tools/flashmob-sidecar/README.md WIP branch for Flash-MoE-SSD

Anemll

29,421 görüntüleme • 3 ay önce

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.
4:15

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.

antirez

269,779 görüntüleme • 1 ay önce

WIP: First attempt to speed up prefill for Flash-MoE. Original repo did token-by-token without streamed experts. Added: Batched linear attention + batched full attention (Flash Attention style) with custom Metal kernels. Without experts: 6.2x faster prefill (11 -> 68 tok/s) With experts at full-attn layers only: 1.9x faster (11 -> 20.5 tok/s) — same output quality Qwen3.5-397B, 4-bit, 209GB, M5 Max 128GB 1/3
0:57

WIP: First attempt to speed up prefill for Flash-MoE. Original repo did token-by-token without streamed experts. Added: Batched linear attention + batched full attention (Flash Attention style) with custom Metal kernels. Without experts: 6.2x faster prefill (11 -> 68 tok/s) With experts at full-attn layers only: 1.9x faster (11 -> 20.5 tok/s) — same output quality Qwen3.5-397B, 4-bit, 209GB, M5 Max 128GB 1/3

Anemll

19,572 görüntüleme • 3 ay önce

I'm now used to this speed 🤣 Running DeepSeek-V4-Flash-IQ2XXS on my single 3090 and 64GB RAM tg is fine, its the prefill that actually hurts.
1:04

I'm now used to this speed 🤣 Running DeepSeek-V4-Flash-IQ2XXS on my single 3090 and 64GB RAM tg is fine, its the prefill that actually hurts.

AJ

20,999 görüntüleme • 10 gün önce

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.
1:24

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.

antirez

169,759 görüntüleme • 1 ay önce

DeepSeek V4 Flash & V4 Pro Official API Speed Test V4 Flash: 80.63 tok/s V4 Pro: 36.72 tok/s
0:22

DeepSeek V4 Flash & V4 Pro Official API Speed Test V4 Flash: 80.63 tok/s V4 Pro: 36.72 tok/s

stevibe

17,555 görüntüleme • 2 ay önce

I can't believe this works, but I got DeepSeek-V4-Flash (284B params) running on a Raspberry Pi 5 (8GB edition) at >1tok/s @ ~8W during full-tilt inference! It uses an untouched copy of antirez's GGUF. Took 160+ experiments over 5 days between GPT-5.5 xhigh and Opus 4.8 max.
0:32

I can't believe this works, but I got DeepSeek-V4-Flash (284B params) running on a Raspberry Pi 5 (8GB edition) at >1tok/s @ ~8W during full-tilt inference! It uses an untouched copy of antirez's GGUF. Took 160+ experiments over 5 days between GPT-5.5 xhigh and Opus 4.8 max.

Dan Woods

184,322 görüntüleme • 1 ay önce

For anyone wandering what does it mean to run ds4-agent locally on an M5 Max using DeepSeek V4 Flash q2-imatrix gguf model. Here's a video of ds4 updating itself, adding a way to leverage HF_HOME for gguf models. Future of Local AI is bright!
2:31

For anyone wandering what does it mean to run ds4-agent locally on an M5 Max using DeepSeek V4 Flash q2-imatrix gguf model. Here's a video of ds4 updating itself, adding a way to leverage HF_HOME for gguf models. Future of Local AI is bright!

Ivan Fioravanti ᯅ

30,319 görüntüleme • 1 ay önce

FREE DeepSeek V4 Pro and V4 Flash API. Runtime by Bad Theory Labs just dropped it. Same platform, new free model, same 2-minute setup. > Runtime already gave you free Claude Opus 4.8 and GPT 5.5 access. Now they added DeepSeek V4 Pro and V4 Flash to the free tier. Absolutely FREE, YES. Same exact flow as before: → Go to runtime(.)badtheorylabs(.)com → Sign up with Google → Fill in your details during onboarding → Free balance lands automatically(But you don't need it to use DeepSeek - it's completely free right now) → Grab your API key from the dashboard → Plug into OpenCode, OpenClaw, or Hermes - done DeepSeek V4 Pro is one of the strongest coding and reasoning models available right now. Free tier also includes btl-2 - auto-routes between GLM, OpenAI, Anthropic, and Gemma. 10 million tokens per month at zero cost. 344 models total. One OpenAI-compatible endpoint. No code rebuild, no card, no limits hunting. >> Just start using the API, and when choosing a provider, select DeepSeek I tested it myself - fully working. Bookmark this before the free tier fills up.
0:23

FREE DeepSeek V4 Pro and V4 Flash API. Runtime by Bad Theory Labs just dropped it. Same platform, new free model, same 2-minute setup. > Runtime already gave you free Claude Opus 4.8 and GPT 5.5 access. Now they added DeepSeek V4 Pro and V4 Flash to the free tier. Absolutely FREE, YES. Same exact flow as before: → Go to runtime(.)badtheorylabs(.)com → Sign up with Google → Fill in your details during onboarding → Free balance lands automatically(But you don't need it to use DeepSeek - it's completely free right now) → Grab your API key from the dashboard → Plug into OpenCode, OpenClaw, or Hermes - done DeepSeek V4 Pro is one of the strongest coding and reasoning models available right now. Free tier also includes btl-2 - auto-routes between GLM, OpenAI, Anthropic, and Gemma. 10 million tokens per month at zero cost. 344 models total. One OpenAI-compatible endpoint. No code rebuild, no card, no limits hunting. >> Just start using the API, and when choosing a provider, select DeepSeek I tested it myself - fully working. Bookmark this before the free tier fills up.

Atenov int.

26,920 görüntüleme • 19 gün önce

DeepSeek-V4-Flash-Spark and Spark-Mini were born today. 1 command setup to Pi / vllm-studio / docker deployment. All this is running local on a single DGX Spark.
3:57

DeepSeek-V4-Flash-Spark and Spark-Mini were born today. 1 command setup to Pi / vllm-studio / docker deployment. All this is running local on a single DGX Spark.

0xSero

20,019 görüntüleme • 1 ay önce

DeepSeek V4 Flash running on the DGX Station.
0:37

DeepSeek V4 Flash running on the DGX Station.

Matthew Berman

70,555 görüntüleme • 2 ay önce

Deepseek-V4-Flash helping me setup Nvidia's Dynamo for disaggregated inference. I have really gotten this model to be a daily driver now. It's really strong at agentic workflows and a decent programmer. For all my side stuff, it's local deepseek now Claude sub cancelled wdyt
0:41

Deepseek-V4-Flash helping me setup Nvidia's Dynamo for disaggregated inference. I have really gotten this model to be a daily driver now. It's really strong at agentic workflows and a decent programmer. For all my side stuff, it's local deepseek now Claude sub cancelled wdyt

0xSero

24,469 görüntüleme • 1 ay önce

Just asked DeepSeek V4 Pro to generate a macOS clone... and uh... it tried. Mid.
0:50

Just asked DeepSeek V4 Pro to generate a macOS clone... and uh... it tried. Mid.

WorldofAI

50,772 görüntüleme • 2 ay önce

DeepSeek V4 ships with two variants allowing it to hit 1M Context. Kimbo @ ICML26 breaks down the attention changes and the Mega MOE speeding up compute.
0:44

DeepSeek V4 ships with two variants allowing it to hit 1M Context. Kimbo @ ICML26 breaks down the attention changes and the Mega MOE speeding up compute.

SemiAnalysis

39,693 görüntüleme • 4 gün önce

We added 4 major open weight models in April. These powerful models are available for encrypted inference in the Maple apps, web, and API: - DeepSeek DeepSeek V4 - Kimi.ai Kimi K2.6 - Z.ai GLM 5.1 - Google Gemma Gemma 4 Power and privacy in one spot.
0:50

We added 4 major open weight models in April. These powerful models are available for encrypted inference in the Maple apps, web, and API: - DeepSeek DeepSeek V4 - Kimi.ai Kimi K2.6 - Z.ai GLM 5.1 - Google Gemma Gemma 4 Power and privacy in one spot.

Maple

11,640 görüntüleme • 2 ay önce

Halo working live. A P2P, permissionless AI protocol for humans and agents. x402 payments per prompt and per tool call to Messari CoinGecko and web data tools. Running Deepseek V4 Flash from an operator serving it via his Nous Research Hermes agent, already earning on inference. All on Base
3:40

Halo working live. A P2P, permissionless AI protocol for humans and agents. x402 payments per prompt and per tool call to Messari CoinGecko and web data tools. Running Deepseek V4 Flash from an operator serving it via his Nous Research Hermes agent, already earning on inference. All on Base

Warden

39,287 görüntüleme • 1 ay önce

I’ve been asked if external SSD works ? Here is M4 Pro 24GB running MinMax 2.7 @ 7.7 fps Unsloth AI quant IQ2_XXS @ 73GB MOE_TOPK=4, --moe-slot-bank 48, It's using ACASIS USB4v2 80 Gbps enclosure with “budget” T710 1TB Gen5 SSD over TB5 connection. I’m also testing different enclosure and SSDs. It seems QD1/Q1T1 random access is the most critical. Llama.cpp experimental fork/branch with MinMax-2.7: Quants and benchmarks are here: ACASIS Official
2:51

I’ve been asked if external SSD works ? Here is M4 Pro 24GB running MinMax 2.7 @ 7.7 fps Unsloth AI quant IQ2_XXS @ 73GB MOE_TOPK=4, --moe-slot-bank 48, It's using ACASIS USB4v2 80 Gbps enclosure with “budget” T710 1TB Gen5 SSD over TB5 connection. I’m also testing different enclosure and SSDs. It seems QD1/Q1T1 random access is the most critical. Llama.cpp experimental fork/branch with MinMax-2.7: Quants and benchmarks are here: ACASIS Official

Anemll

18,597 görüntüleme • 2 ay önce

DeepSeek V4 Flash vs V4 Pro, 4 canvas tests run. I'm a dev, and honestly DeepSeek was never on my list for coding tasks. After seeing these results, I might need to reconsider. 🌳 Tree: Flash actually beat Pro: denser canopy, better leaf distribution 🌌 Night sky: Pro wins, shooting stars, constellations, Milky Way band all there 🐟 Fish boids: Pro is stunning: proper caustics, tight schooling, real underwater feel 🏠 House builder: first model we've tested that knows how to actually build a rooftop DeepSeek coding might be underrated.
0:42

DeepSeek V4 Flash vs V4 Pro, 4 canvas tests run. I'm a dev, and honestly DeepSeek was never on my list for coding tasks. After seeing these results, I might need to reconsider. 🌳 Tree: Flash actually beat Pro: denser canopy, better leaf distribution 🌌 Night sky: Pro wins, shooting stars, constellations, Milky Way band all there 🐟 Fish boids: Pro is stunning: proper caustics, tight schooling, real underwater feel 🏠 House builder: first model we've tested that knows how to actually build a rooftop DeepSeek coding might be underrated.

stevibe

56,270 görüntüleme • 2 ay önce

Want to give v4 a test drive? 🚀 Available now on iOS, and Android. Make more free v4 songs today.
0:27

Want to give v4 a test drive? 🚀 Available now on iOS, and Android. Make more free v4 songs today.

Suno

20,567 görüntüleme • 1 yıl önce

DeepSeek-V4-Flash powering 4 parallel agents on Pi (by Mario Zechner) 🚀 Running on M3 Ultra at ~30-34 tok/s and 160-187GB peak URAM using MLX-LM. Special shoutout to clandestine.eth 🦇🔊, Pedro Cuenca, Tarjei Mandt, Ivan Fioravanti ᯅ and others for helping optimize and shape this PR. PR:
1:29

DeepSeek-V4-Flash powering 4 parallel agents on Pi (by Mario Zechner) 🚀 Running on M3 Ultra at ~30-34 tok/s and 160-187GB peak URAM using MLX-LM. Special shoutout to clandestine.eth 🦇🔊, Pedro Cuenca, Tarjei Mandt, Ivan Fioravanti ᯅ and others for helping optimize and shape this PR. PR:

Prince Canuma

109,941 görüntüleme • 2 ay önce