正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Initial DeepSeek v4 and v4 Flash support added to anemll-flash-llama.cpp! 🚀 M5 MAX 128GB can run full DeepSeek-V4 with 1.6T params in original FP8/FP4 weights from SSD without requantization! One-click scripts to convert HF safetensors to dense + MoE sidecar (no GGUF needed) Inference and server examples Prefill and... show more

Anemll

4,383 subscribers

32,250 次观看 • 3 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

~ 6.5 - 6.7 t/s for GLM 5.1 on M5 Max 128GB Added “Dense” model export, now model load is only 5s ! Experts are streaming from SSD, so we do not pre-load it. Added direct SSD->Slot memory path, removed prefetch... Many dead end experiments. See Export a “dense-only GGUF” and “Fast path ” in tools/flashmob-sidecar/README.md WIP branch for Flash-MoE-SSD

~ 6.5 - 6.7 t/s for GLM 5.1 on M5 Max 128GB Added “Dense” model export, now model load is only 5s ! Experts are streaming from SSD, so we do not pre-load it. Added direct SSD->Slot memory path, removed prefetch... Many dead end experiments. See Export a “dense-only GGUF” and “Fast path ” in tools/flashmob-sidecar/README.md WIP branch for Flash-MoE-SSD

Anemll

29,421 次观看 • 4 个月前

Added web search to Papers with Code's upcoming chat interface This runs on Hugging Face Inference Providers via Baseten (new DeepSeek-V4 Flash), using Exa's MCP server, running inside a Modal container Drop your HF username, and I'll add you, eager for feedback :)

Added web search to Papers with Code's upcoming chat interface This runs on Hugging Face Inference Providers via Baseten (new DeepSeek-V4 Flash), using Exa's MCP server, running inside a Modal container Drop your HF username, and I'll add you, eager for feedback :)

Niels Rogge

19,968 次观看 • 12 天前

WIP: First attempt to speed up prefill for Flash-MoE. Original repo did token-by-token without streamed experts. Added: Batched linear attention + batched full attention (Flash Attention style) with custom Metal kernels. Without experts: 6.2x faster prefill (11 -> 68 tok/s) With experts at full-attn layers only: 1.9x faster (11 -> 20.5 tok/s) — same output quality Qwen3.5-397B, 4-bit, 209GB, M5 Max 128GB 1/3

WIP: First attempt to speed up prefill for Flash-MoE. Original repo did token-by-token without streamed experts. Added: Batched linear attention + batched full attention (Flash Attention style) with custom Metal kernels. Without experts: 6.2x faster prefill (11 -> 68 tok/s) With experts at full-attn layers only: 1.9x faster (11 -> 20.5 tok/s) — same output quality Qwen3.5-397B, 4-bit, 209GB, M5 Max 128GB 1/3

Anemll

19,572 次观看 • 4 个月前

DeepSeek V4 Flash 0731 in Hermes Agent and one prompt, took 32 minutes and cost 0.07$, this model is so cheap to the point where 2 dollars can last you a full day.

DeepSeek V4 Flash 0731 in Hermes Agent and one prompt, took 32 minutes and cost 0.07$, this model is so cheap to the point where 2 dollars can last you a full day.

Elshayib

676,502 次观看 • 19 天前

Timelapse #177 (14 hrs) - Picked up an RTX Pro 6000 Blackwell and wired in another 1000W PSU - running two of them now - DeepSeek v4 Flash running locally on both GPUs - Distilling DeepSeek v4 Flash down into Nanbeige (still cooking) - Kernel bench runs for DeepSeek v4 Flash and Qwen 3.8 Max - Got it into chat but couldn't reliably get it into the OMP harness yet - Private contract work - days like this are stuff I actually want to build. and when it's the high-intensity grind, i still have to show up and get it done

Timelapse #177 (14 hrs) - Picked up an RTX Pro 6000 Blackwell and wired in another 1000W PSU - running two of them now - DeepSeek v4 Flash running locally on both GPUs - Distilling DeepSeek v4 Flash down into Nanbeige (still cooking) - Kernel bench runs for DeepSeek v4 Flash and Qwen 3.8 Max - Got it into chat but couldn't reliably get it into the OMP harness yet - Private contract work - days like this are stuff I actually want to build. and when it's the high-intensity grind, i still have to show up and get it done

Elliot Arledge

17,401 次观看 • 15 天前

FREE DeepSeek V4 Pro and V4 Flash API. Runtime by Bad Theory Labs just dropped it. Same platform, new free model, same 2-minute setup. > Runtime already gave you free Claude Opus 4.8 and GPT 5.5 access. Now they added DeepSeek V4 Pro and V4 Flash to the free tier. Absolutely FREE, YES. Same exact flow as before: → Go to runtime(.)badtheorylabs(.)com → Sign up with Google → Fill in your details during onboarding → Free balance lands automatically(But you don't need it to use DeepSeek - it's completely free right now) → Grab your API key from the dashboard → Plug into OpenCode, OpenClaw, or Hermes - done DeepSeek V4 Pro is one of the strongest coding and reasoning models available right now. Free tier also includes btl-2 - auto-routes between GLM, OpenAI, Anthropic, and Gemma. 10 million tokens per month at zero cost. 344 models total. One OpenAI-compatible endpoint. No code rebuild, no card, no limits hunting. >> Just start using the API, and when choosing a provider, select DeepSeek I tested it myself - fully working. Bookmark this before the free tier fills up.

FREE DeepSeek V4 Pro and V4 Flash API. Runtime by Bad Theory Labs just dropped it. Same platform, new free model, same 2-minute setup. > Runtime already gave you free Claude Opus 4.8 and GPT 5.5 access. Now they added DeepSeek V4 Pro and V4 Flash to the free tier. Absolutely FREE, YES. Same exact flow as before: → Go to runtime(.)badtheorylabs(.)com → Sign up with Google → Fill in your details during onboarding → Free balance lands automatically(But you don't need it to use DeepSeek - it's completely free right now) → Grab your API key from the dashboard → Plug into OpenCode, OpenClaw, or Hermes - done DeepSeek V4 Pro is one of the strongest coding and reasoning models available right now. Free tier also includes btl-2 - auto-routes between GLM, OpenAI, Anthropic, and Gemma. 10 million tokens per month at zero cost. 344 models total. One OpenAI-compatible endpoint. No code rebuild, no card, no limits hunting. >> Just start using the API, and when choosing a provider, select DeepSeek I tested it myself - fully working. Bookmark this before the free tier fills up.

Atenov int.

26,920 次观看 • 1 个月前

This is probably my most important post about DeepSeek V4 Flash. Do NOT judge this model after testing only one harness. Pi, Claude Code, and OpenCode produced dramatically different results—and OpenCode completely changed my verdict. You really need to read the details below.

This is probably my most important post about DeepSeek V4 Flash. Do NOT judge this model after testing only one harness. Pi, Claude Code, and OpenCode produced dramatically different results—and OpenCode completely changed my verdict. You really need to read the details below.

OmedTheVibeCoder

68,210 次观看 • 18 天前

DwarfStar DeepSeek v4 Flash 0731 pre-M5 kernel keeps improving 💪🏻 - prefill +11% - decode +10% - single request no DSpark/MTP. - 100% same logics between baseline and now. Today is QA like crazy. I hope to file PR by EOD antirez Here are results on the M3 Ultra and a video now vs baseline using ds4-eval.

DwarfStar DeepSeek v4 Flash 0731 pre-M5 kernel keeps improving 💪🏻 - prefill +11% - decode +10% - single request no DSpark/MTP. - 100% same logics between baseline and now. Today is QA like crazy. I hope to file PR by EOD antirez Here are results on the M3 Ultra and a video now vs baseline using ds4-eval.

Ivan Fioravanti ᯅ

17,121 次观看 • 12 天前

SGLang now supports DSpark, enabling confidence-driven, variable-length verification for speculative decoding 🎉 DSpark addresses a key bottleneck under load: instead of verifying every draft token, it verifies only where the draft model is confident, so the gains hold even as batch size scales. We heavily optimized variable-length verification in SGLang. Across batch sizes 1 to 256, DSpark gives the best throughput/latency tradeoff on DeepSeek-V4-Flash, ahead of both MTP and non-spec. At high concurrency, dynamic scheduling provides up to ~20% higher throughput compared to a fixed budget, while maintaining high verification quality across workloads. With fused kernels and zero-overhead scheduling, DeepSeek-V4-Pro reaches 383.7 tok/s at B=1 on B300. DSpark is now available in SGLang with support for Qwen3 and DeepSeek-V4. Thanks DeepSeek for open-sourcing! Blog with full technical details and commands to run below 👇

SGLang now supports DSpark, enabling confidence-driven, variable-length verification for speculative decoding 🎉 DSpark addresses a key bottleneck under load: instead of verifying every draft token, it verifies only where the draft model is confident, so the gains hold even as batch size scales. We heavily optimized variable-length verification in SGLang. Across batch sizes 1 to 256, DSpark gives the best throughput/latency tradeoff on DeepSeek-V4-Flash, ahead of both MTP and non-spec. At high concurrency, dynamic scheduling provides up to ~20% higher throughput compared to a fixed budget, while maintaining high verification quality across workloads. With fused kernels and zero-overhead scheduling, DeepSeek-V4-Pro reaches 383.7 tok/s at B=1 on B300. DSpark is now available in SGLang with support for Qwen3 and DeepSeek-V4. Thanks DeepSeek for open-sourcing! Blog with full technical details and commands to run below 👇

LMSYS Org

169,905 次观看 • 1 个月前

I’ve been asked if external SSD works ? Here is M4 Pro 24GB running MinMax 2.7 @ 7.7 fps Unsloth AI quant IQ2_XXS @ 73GB MOE_TOPK=4, --moe-slot-bank 48, It's using ACASIS USB4v2 80 Gbps enclosure with “budget” T710 1TB Gen5 SSD over TB5 connection. I’m also testing different enclosure and SSDs. It seems QD1/Q1T1 random access is the most critical. Llama.cpp experimental fork/branch with MinMax-2.7: Quants and benchmarks are here: ACASIS Official

I’ve been asked if external SSD works ? Here is M4 Pro 24GB running MinMax 2.7 @ 7.7 fps Unsloth AI quant IQ2_XXS @ 73GB MOE_TOPK=4, --moe-slot-bank 48, It's using ACASIS USB4v2 80 Gbps enclosure with “budget” T710 1TB Gen5 SSD over TB5 connection. I’m also testing different enclosure and SSDs. It seems QD1/Q1T1 random access is the most critical. Llama.cpp experimental fork/branch with MinMax-2.7: Quants and benchmarks are here: ACASIS Official

Anemll

18,597 次观看 • 4 个月前

GPT-5.6 LUNA AND DEEPSEEK V4 FLASH ARE BOTH FREE RIGHT NOW -- ONE IN A CHATBOT, THE OTHER IN A BROWSER IDE • ChatGPT, the chatbot > GPT-5.6 Luna is now the default on Free and Go: > Unlimited text chats and a Think button land next week. • MonkeyCode, the browser IDE > 30M tokens a day, resets daily, no card: > DeepSeek V4 Flash, Qwen, GLM, Kimi and MiniMax behind it. > Cloud workspace at 1 vCPU and 4GB, nothing to install. Luna posts 62% fewer factual errors than the model it replaces. MonkeyCode is open source from Chaitin and runs on their cloud, so aim it at side projects. One is live today -> The other arrives next week.

GPT-5.6 LUNA AND DEEPSEEK V4 FLASH ARE BOTH FREE RIGHT NOW -- ONE IN A CHATBOT, THE OTHER IN A BROWSER IDE • ChatGPT, the chatbot > GPT-5.6 Luna is now the default on Free and Go: > Unlimited text chats and a Think button land next week. • MonkeyCode, the browser IDE > 30M tokens a day, resets daily, no card: > DeepSeek V4 Flash, Qwen, GLM, Kimi and MiniMax behind it. > Cloud workspace at 1 vCPU and 4GB, nothing to install. Luna posts 62% fewer factual errors than the model it replaces. MonkeyCode is open source from Chaitin and runs on their cloud, so aim it at side projects. One is live today -> The other arrives next week.

slash1s

33,804 次观看 • 11 天前

$50 IN FREE CREDITS FOR GLM 5.2, KIMI K3 AND DEEPSEEK V4 PRO, ON A CODE THE COMPANY PUBLISHED ITSELF • ai&, the inference layer > Frontier models at up to 80% lower cost: > GLM 5.2, Kimi K3, Kimi K2.7 Code, DeepSeek V4 Pro and V4 Flash, plus Qwen3.6 27B, Gemma 4 31B and gpt-oss-120b. > OpenAI-compatible endpoint, so it drops into any agent or IDE you already run. • where the discount comes from > They route across AMD, NVIDIA and Tenstorrent, sending each workload to the silicon that suits it. > Everyone else tunes on one kind of chip, so the saving is architectural rather than promotional. Claiming it -> Credits in the top right, pick $50, apply "UNITABETAI" at checkout. The code has been live since their June launch, so run it on your own account before you plan around it ↓

$50 IN FREE CREDITS FOR GLM 5.2, KIMI K3 AND DEEPSEEK V4 PRO, ON A CODE THE COMPANY PUBLISHED ITSELF • ai&, the inference layer > Frontier models at up to 80% lower cost: > GLM 5.2, Kimi K3, Kimi K2.7 Code, DeepSeek V4 Pro and V4 Flash, plus Qwen3.6 27B, Gemma 4 31B and gpt-oss-120b. > OpenAI-compatible endpoint, so it drops into any agent or IDE you already run. • where the discount comes from > They route across AMD, NVIDIA and Tenstorrent, sending each workload to the silicon that suits it. > Everyone else tunes on one kind of chip, so the saving is architectural rather than promotional. Claiming it -> Credits in the top right, pick $50, apply "UNITABETAI" at checkout. The code has been live since their June launch, so run it on your own account before you plan around it ↓

slash1s

26,238 次观看 • 10 天前

DeepSeek V4 Flash completely failed this test. Compared to Opus 4.8, GPT-5.6 Sol, Kimi K3, and the other current top models, it’s honestly not good. The gap is massive. Against Luna, however, it performs similarly well. Both have different strengths and weaknesses, so I still need to test them more. The weird part: the Pi harness performed worse than OpenCode for DeepSeek. More tests are coming, guys. Don’t worry

DeepSeek V4 Flash completely failed this test. Compared to Opus 4.8, GPT-5.6 Sol, Kimi K3, and the other current top models, it’s honestly not good. The gap is massive. Against Luna, however, it performs similarly well. Both have different strengths and weaknesses, so I still need to test them more. The weird part: the Pi harness performed worse than OpenCode for DeepSeek. More tests are coming, guys. Don’t worry

OmedTheVibeCoder

25,980 次观看 • 19 天前

DGX Station: this is just the start of what it is possible to do (and why I claim DwarfStar could be *the* Station inference engine). DeepSeek v4 PRO Q2 with routed experts split among VRAM / RAM with kernels optimized for this peculiar setup. 45 t/s but can go faster.

DGX Station: this is just the start of what it is possible to do (and why I claim DwarfStar could be the Station inference engine). DeepSeek v4 PRO Q2 with routed experts split among VRAM / RAM with kernels optimized for this peculiar setup. 45 t/s but can go faster.

antirez

41,080 次观看 • 3 天前

holy sh*t this is f**king insane I cancelled my $200/mo Claude for this I replaced opus 5 with deepseek v4-flash and someone figured out to how to use them as subagents on codex [it takes 3 mins to set up here is how] 1. install this repo called 'model-router' 2. toggle subagent models > 'all selected models' 3. that's it

holy sh*t this is f**king insane I cancelled my $200/mo Claude for this I replaced opus 5 with deepseek v4-flash and someone figured out to how to use them as subagents on codex [it takes 3 mins to set up here is how] 1. install this repo called 'model-router' 2. toggle subagent models > 'all selected models' 3. that's it

Avid

414,365 次观看 • 12 天前

HERMES DESKTOP APP JUST GOT A NEW ULTRA MODE FOR DEEPSEEK V4 PRO – AND IT COSTS ABOUT $1 A DAY TO RUN If you're using Hermes desktop app, you're probably already running DeepSeek – it's one of the top models available. The update: → DeepSeek V4 Pro now has thinking mode, fast mode, and a brand new ultra mode → There's also an extra high mode, but the real upgrade is ultra The actual cost: roughly $1/day for a normal workflow. That's about $20-25/month – basically the same as before, just with a significantly faster mode unlocked. The verdict after a full day of testing: it flies through tool calling. Noticeably faster, same price range. If you're already paying for DeepSeek through Hermes, this is a free upgrade sitting in your settings right now. Update Hermes, grab the DeepSeek API if you don't have it yet, and try V4 Pro on ultra. Bookmark this, you'll want to come back to it later.

SCOTTY BEAM

24,032 次观看 • 1 个月前

I have been testing DeepSeek-V4-Pro with the Pi coding agent. I am mindblown by how well it works out of the box. A few notes: I spent a few hours building an LLM wiki with an agent powered entirely by DeepSeek-V4-Pro on Fireworks inference. This is the first time I feel like there is an open-weight model that can reason at the level of Claude and Codex. And it does this in a cost-effective way with support for 1M context length. To be clear, I am using DeepSeek-V4-Pro inside of Pi without any special configuration. It works out of the box. It's exciting that there is a model that can just be plugged into a basic harness like Pi, and it just works. I've never seen that before. Most models require lots of configuration and setup. DeepSeek's DeepSeek-V4-Pro is clearly good at agentic coding (probably the best from the open-weight models), but the model is also great on knowledge-intensive tasks where reasoning matters. The agent pulled agentic engineering best practices from different company docs (Anthropic, OpenAI, Google, Stripe, Meta, Modal, DeepSeek, Mistral, Cohere), searched and digested Reddit and HN threads, summarized arxiv papers, and surfaced trending GitHub repos. Then it distilled everything into actionable tips across categories. I love the Wiki it built. The quality is really good. Here is a snapshot of what the wiki looks like: DeepSeek-V4-Pro handled the task without breaking stride. Multi-step research queries, code generation for scaffolding, context-heavy reasoning across disparate sources. For coding specifically, this is the first open-weight model that genuinely feels like a Codex or Claude Code experience. It compares in capability and actual multi-turn agentic work. What made the loop feel so responsive was Fireworks' inference speed (the fastest in the market) and the fact that they actually validate models at the systems level before shipping. No corrupted reasoning traces. Just fast, reliable iteration. The hybrid CSA and HCA attention design cuts KV cache to just 10% and inference FLOPs by nearly 4x at 1M-token context. This is what makes the agent loop actually fast and cheap enough to run in practice. For devs who've been watching open-weight models close the gap but haven't found one that actually delivers in practice, this is the closest I've seen. Try it here:

I have been testing DeepSeek-V4-Pro with the Pi coding agent. I am mindblown by how well it works out of the box. A few notes: I spent a few hours building an LLM wiki with an agent powered entirely by DeepSeek-V4-Pro on Fireworks inference. This is the first time I feel like there is an open-weight model that can reason at the level of Claude and Codex. And it does this in a cost-effective way with support for 1M context length. To be clear, I am using DeepSeek-V4-Pro inside of Pi without any special configuration. It works out of the box. It's exciting that there is a model that can just be plugged into a basic harness like Pi, and it just works. I've never seen that before. Most models require lots of configuration and setup. DeepSeek's DeepSeek-V4-Pro is clearly good at agentic coding (probably the best from the open-weight models), but the model is also great on knowledge-intensive tasks where reasoning matters. The agent pulled agentic engineering best practices from different company docs (Anthropic, OpenAI, Google, Stripe, Meta, Modal, DeepSeek, Mistral, Cohere), searched and digested Reddit and HN threads, summarized arxiv papers, and surfaced trending GitHub repos. Then it distilled everything into actionable tips across categories. I love the Wiki it built. The quality is really good. Here is a snapshot of what the wiki looks like: DeepSeek-V4-Pro handled the task without breaking stride. Multi-step research queries, code generation for scaffolding, context-heavy reasoning across disparate sources. For coding specifically, this is the first open-weight model that genuinely feels like a Codex or Claude Code experience. It compares in capability and actual multi-turn agentic work. What made the loop feel so responsive was Fireworks' inference speed (the fastest in the market) and the fact that they actually validate models at the systems level before shipping. No corrupted reasoning traces. Just fast, reliable iteration. The hybrid CSA and HCA attention design cuts KV cache to just 10% and inference FLOPs by nearly 4x at 1M-token context. This is what makes the agent loop actually fast and cheap enough to run in practice. For devs who've been watching open-weight models close the gap but haven't found one that actually delivers in practice, this is the closest I've seen. Try it here:

elvis

60,091 次观看 • 3 个月前

Where there's a will, there's a way 😎 DwarfStar DeepSeek V4 Flash 0731 mxfp4 on M3 Ultra single request no MPT/DSpark ~41 toks/s decode!!! 🚀 40 tps barrier broken! Same logits, 100% same result: this is The Mandatory Rule for any agents improving kernels that I think everyone should follow. Speed without mathematical precision is useless. Thanks 20% Opus 5 and 80% Kimi.ai K3 (what a model!) And just looking at the thinking trace of K3 you can learn tons of stuff! Here usually I have a separate chat with K3 and when I see something I want to dig deeper or ask about from the main thinking trace I do and in some cases I stop the experiment and pivot immediately to something else. Now let's see if I can apply learned lessons to the M5 kernel! 🚀 Branch for any braves soul willing to test it:

Where there's a will, there's a way 😎 DwarfStar DeepSeek V4 Flash 0731 mxfp4 on M3 Ultra single request no MPT/DSpark ~41 toks/s decode!!! 🚀 40 tps barrier broken! Same logits, 100% same result: this is The Mandatory Rule for any agents improving kernels that I think everyone should follow. Speed without mathematical precision is useless. Thanks 20% Opus 5 and 80% Kimi.ai K3 (what a model!) And just looking at the thinking trace of K3 you can learn tons of stuff! Here usually I have a separate chat with K3 and when I see something I want to dig deeper or ask about from the main thinking trace I do and in some cases I stop the experiment and pivot immediately to something else. Now let's see if I can apply learned lessons to the M5 kernel! 🚀 Branch for any braves soul willing to test it:

Ivan Fioravanti ᯅ

18,381 次观看 • 12 天前

Laguna S 2.1 is built to run locally. We heard the ask for a serious open model in the 80–120B range. Laguna S 2.1 is our answer, and we want it to redefine what this size class can do. Weights ship in BF16, FP8, INT4 and NVFP4, with official GGUF and MLX conversions, plus official DFlash draft models for faster inference. Day-one support across llama.cpp, ollama, Valerie Lee Make Up, SGLang, NVIDIA TRT-LLM, Hugging Face transformers and ZML. Watch it run on an NVIDIA AI DGX Spark.

Laguna S 2.1 is built to run locally. We heard the ask for a serious open model in the 80–120B range. Laguna S 2.1 is our answer, and we want it to redefine what this size class can do. Weights ship in BF16, FP8, INT4 and NVFP4, with official GGUF and MLX conversions, plus official DFlash draft models for faster inference. Day-one support across llama.cpp, ollama, Valerie Lee Make Up, SGLang, NVIDIA TRT-LLM, Hugging Face transformers and ZML. Watch it run on an NVIDIA AI DGX Spark.

Poolside

31,242 次观看 • 29 天前

Apple’s “LLM in a Flash” is definitely worth checking out. Going to 2-bit for the shared-expert MLP means disk I/O is no longer dominant. 14–15 tok/s from SSD is still wild for a ~400B MoE model streamed from storage. Qwen3.5-397B-A17B Credit: Dan Woods

Apple’s “LLM in a Flash” is definitely worth checking out. Going to 2-bit for the shared-expert MLP means disk I/O is no longer dominant. 14–15 tok/s from SSD is still wild for a ~400B MoE model streamed from storage. Qwen3.5-397B-A17B Credit: Dan Woods

Anemll

35,664 次观看 • 5 个月前