Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

DeepSeek's open-sourced engine hits 1.7M tok/s 🐳 DeepSeek released DeepInfer, which serves V4.1 Flash in production, with 325 tok/s for one chat and about 1.7M tok/s across 64 Nvidia Blackwell GPUs Run AI models locally ->

atomic_chat_hq's profile picture

atomic.chat

20,216 subscribers

30,817 Aufrufe • vor 26 Tagen •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Fruit-fly connectome optimized inference for DeepSeek-V4.1-Flash on 2× NVIDIA GB300 DGX Stations! It impressively hits 56k tok/s prefill and 201.7 tok/s c1 decode. Well done little dude.
0:14

Fruit-fly connectome optimized inference for DeepSeek-V4.1-Flash on 2× NVIDIA GB300 DGX Stations! It impressively hits 56k tok/s prefill and 201.7 tok/s c1 decode. Well done little dude.

catid

12,293 Aufrufe • vor 2 Monaten

Okay, I guess I didn't waste all that money after all Deepseek V4.1 Flash 304 tok/s code, 135 tok/s prose Zero optimization, will have it max optimized by the morning.
0:17

Okay, I guess I didn't waste all that money after all Deepseek V4.1 Flash 304 tok/s code, 135 tok/s prose Zero optimization, will have it max optimized by the morning.

Killy

44,653 Aufrufe • vor 2 Monaten

One task, two models. GLM 5.3 Flash running slower at 25 tok/s prose Qwen3.8 Flash running faster at 47 tok/s prose Who will finish the task first?
0:14

One task, two models. GLM 5.3 Flash running slower at 25 tok/s prose Qwen3.8 Flash running faster at 47 tok/s prose Who will finish the task first?

Mia

27,481 Aufrufe • vor 2 Monaten

This is awesome. Thanks Mia for this Was able to run with the RTX 3090 use this .env config DRAFT=mtp CONTEXT_SIZE=262144 CACHE_QUANT=8,4 GPU_MEM_GB=22 262K context: - Run 1: 65.98 tok/s - Run 2: 63.23 tok/s - Run 3: 64.41 tok/s Average: ~64.5 tok/s.
0:12

This is awesome. Thanks Mia for this Was able to run with the RTX 3090 use this .env config DRAFT=mtp CONTEXT_SIZE=262144 CACHE_QUANT=8,4 GPU_MEM_GB=22 262K context: - Run 1: 65.98 tok/s - Run 2: 63.23 tok/s - Run 3: 64.41 tok/s Average: ~64.5 tok/s.

Melvin Vivas

99,403 Aufrufe • vor 2 Monaten

Deepseek v4.1 Flash on 24GB Macbook. I think I reached my limit, 1.36 tok/s. It is far from being usable but the feeling of having a SOTA model on my consumer machine is awesome. I started from 0.74 tok/s and almost double it. It was a great journey.
0:15

Deepseek v4.1 Flash on 24GB Macbook. I think I reached my limit, 1.36 tok/s. It is far from being usable but the feeling of having a SOTA model on my consumer machine is awesome. I started from 0.74 tok/s and almost double it. It was a great journey.

Marco Franzon

90,955 Aufrufe • vor 2 Monaten

4 DGX Sparks running DeepSeek V4.1 Flash 6 concurrent sessions, 170.73 tok/s coding throughput. 🔥 1,200 output tokens in 7.028 seconds. Warm single-stream: 💻 Code: 77.61 tok/s 🧮 Math: 74.08 🧠 Reasoning: 48.75 ✍️ Prose: 33.02 6-stream aggregate: 💻 Code: 170.73 tok/s 🔢 Counting: 291.64 tok/s 552B MoE · MXFP4 · TP4 · vLLM + DSpark + CUDA graphs ✅ Tool calling + vision ✅ 127,480-token retrieval test passed ✅ 300K configured context Video replays the actual recorded streams at 1× speed. Thinking OFF, 200-token cap per coding request. Short-prompt coding throughput Credit to Tech2Wild for the Boot9 recipe. NVIDIA AI
0:10

4 DGX Sparks running DeepSeek V4.1 Flash 6 concurrent sessions, 170.73 tok/s coding throughput. 🔥 1,200 output tokens in 7.028 seconds. Warm single-stream: 💻 Code: 77.61 tok/s 🧮 Math: 74.08 🧠 Reasoning: 48.75 ✍️ Prose: 33.02 6-stream aggregate: 💻 Code: 170.73 tok/s 🔢 Counting: 291.64 tok/s 552B MoE · MXFP4 · TP4 · vLLM + DSpark + CUDA graphs ✅ Tool calling + vision ✅ 127,480-token retrieval test passed ✅ 300K configured context Video replays the actual recorded streams at 1× speed. Thinking OFF, 200-token cap per coding request. Short-prompt coding throughput Credit to Tech2Wild for the Boot9 recipe. NVIDIA AI

Wësche

24,282 Aufrufe • vor 2 Monaten

Space Bunny Alpha mogged by DeepSeek V4.1 Flash 🐳 We asked both to build Minecraft in one HTML file and DeepSeek V4.1 Flash made a forest biome you can walk and dig in while Space Bunny Alpha's ground has holes showing the sky Run AI models locally ->
0:20

Space Bunny Alpha mogged by DeepSeek V4.1 Flash 🐳 We asked both to build Minecraft in one HTML file and DeepSeek V4.1 Flash made a forest biome you can walk and dig in while Space Bunny Alpha's ground has holes showing the sky Run AI models locally ->

atomic.chat

51,067 Aufrufe • vor 1 Monat

NVIDIA just dropped Nemotron-3-Nano:4b — a tiny 2.8GB model. Guess whose hardware runs it the fastest? - RTX 4090: 226 tok/s - RTX 3090: 187 tok/s - Mac Studio M2 Ultra: 86 tok/s - Mac Mini M4: 25 tok/s Home court advantage is real. Also trying a new layout with live performance charts. Lmk what you think!
0:13

NVIDIA just dropped Nemotron-3-Nano:4b — a tiny 2.8GB model. Guess whose hardware runs it the fastest? - RTX 4090: 226 tok/s - RTX 3090: 187 tok/s - Mac Studio M2 Ultra: 86 tok/s - Mac Mini M4: 25 tok/s Home court advantage is real. Also trying a new layout with live performance charts. Lmk what you think!

stevibe

127,672 Aufrufe • vor 8 Monaten

10x speedup: OpenAI's privacy-filter on MLX vs CPU, side by side in your browser. 1,818 tok/s vs 179 tok/s. Real-time PII redaction across English, German, French. On-device. No cloud. Privacy-first. Which model should we port next?
0:15

10x speedup: OpenAI's privacy-filter on MLX vs CPU, side by side in your browser. 1,818 tok/s vs 179 tok/s. Real-time PII redaction across English, German, French. On-device. No cloud. Privacy-first. Which model should we port next?

Maziyar PANAHI

25,329 Aufrufe • vor 6 Monaten

RadixArkのQwen3.8-Flash-Next-NVFP4 つかってFreeTokenをcodexで改造してやっと良きスピード出た Activeが小さいからそれなりに速度出た RAM DDR5 8ch GPU - RTX5090 x1 Prefill 277.5 tok/s Decode 59.13 tok/s - RTX5090 x2 Prefill 619.5 tok/s Decode 83.14 tok/s
0:23

RadixArkのQwen3.8-Flash-Next-NVFP4 つかってFreeTokenをcodexで改造してやっと良きスピード出た Activeが小さいからそれなりに速度出た RAM DDR5 8ch GPU - RTX5090 x1 Prefill 277.5 tok/s Decode 59.13 tok/s - RTX5090 x2 Prefill 619.5 tok/s Decode 83.14 tok/s

ぱぷりか炒め

27,962 Aufrufe • vor 2 Monaten

MiniMax M2.5 casually responding at ~50 tok/s with MLX (M3 Ultra). The model was released one hour ago 🥳
0:14

MiniMax M2.5 casually responding at ~50 tok/s with MLX (M3 Ultra). The model was released one hour ago 🥳

Pedro Cuenca

170,567 Aufrufe • vor 9 Monaten

no one is ready for GPT-5.6 on Cerebras at 750 tok/s
0:13

no one is ready for GPT-5.6 on Cerebras at 750 tok/s

Kai Brokering

203,281 Aufrufe • vor 4 Monaten

BENCHMARKED: LFM3-1.7B by Liquid AI. Liquid AI published 1.6 GB for the KV cache at 128K context. I ran it on my Mac mini and measured 1.6 GB. The whole model at 128K uses 3.15 GB of GPU memory, under 5% of the 64GB. It still generates 80 tok/s. Only 6 of its 24 layers keep a full KV cache. The other 18 use short convolutions or sliding-window attention. llama.cpp (Metal), Q4_K_M, M5 Pro 64GB, median of 3 runs - 512 ctx: 184 tok/s, 0.09s to first token - 4K ctx: 178 tok/s, 0.72s to first token - 32K ctx: 141 tok/s - 128K ctx: 80 tok/s Q8_0 does 125 tok/s at 512, so 4-bit is 47% faster.
0:20

BENCHMARKED: LFM3-1.7B by Liquid AI. Liquid AI published 1.6 GB for the KV cache at 128K context. I ran it on my Mac mini and measured 1.6 GB. The whole model at 128K uses 3.15 GB of GPU memory, under 5% of the 64GB. It still generates 80 tok/s. Only 6 of its 24 layers keep a full KV cache. The other 18 use short convolutions or sliding-window attention. llama.cpp (Metal), Q4_K_M, M5 Pro 64GB, median of 3 runs - 512 ctx: 184 tok/s, 0.09s to first token - 4K ctx: 178 tok/s, 0.72s to first token - 32K ctx: 141 tok/s - 128K ctx: 80 tok/s Q8_0 does 125 tok/s at 512, so 4-bit is 47% faster.

Pratik Gandhi

39,576 Aufrufe • vor 19 Tagen

Day-0 Qwen3.8-27B vs Qwen3.6-27B: the voxel pagoda test. Same prompt, one attempt each, single DGX Spark, both NVFP4. 3.8: 75,291 tok · 52m 56s · 23.7 tok/s (vLLM, native MTP) 3.6: 12,117 tok · 5m 13s · 38.8 tok/s (llama.cpp, DFlash K10) 3.8 thought for 160K characters before writing a line of code.
0:15

Day-0 Qwen3.8-27B vs Qwen3.6-27B: the voxel pagoda test. Same prompt, one attempt each, single DGX Spark, both NVFP4. 3.8: 75,291 tok · 52m 56s · 23.7 tok/s (vLLM, native MTP) 3.6: 12,117 tok · 5m 13s · 38.8 tok/s (llama.cpp, DFlash K10) 3.8 thought for 160K characters before writing a line of code.

Wësche

102,744 Aufrufe • vor 3 Monaten

DeepSeek V4.1 Flash vs Luna at Tesla Roadster 🏎️ DeepSeek V4.1 Flash 1017 re-created the Tesla Roadster event in 3D voxels for $0.89 while GPT-6 Luna's version cost $1.07 Run DeepSeek Flash locally or via API ->
0:16

DeepSeek V4.1 Flash vs Luna at Tesla Roadster 🏎️ DeepSeek V4.1 Flash 1017 re-created the Tesla Roadster event in 3D voxels for $0.89 while GPT-6 Luna's version cost $1.07 Run DeepSeek Flash locally or via API ->

atomic.chat

210,101 Aufrufe • vor 29 Tagen

Two models. One DGX Spark One identical voxel Eiffel Tower prompt. Qwen3.8-27B xhigh: 83K tokens, 41m29s, 33.4 tok/s Ornith 1.5 35B-A3B: 47K tokens, 10m35s, 74.0 tok/s Thinking on, no token ceiling, matched camera. Ornith finished much faster; Qwen spent far more tokens.
0:18

Two models. One DGX Spark One identical voxel Eiffel Tower prompt. Qwen3.8-27B xhigh: 83K tokens, 41m29s, 33.4 tok/s Ornith 1.5 35B-A3B: 47K tokens, 10m35s, 74.0 tok/s Thinking on, no token ceiling, matched camera. Ornith finished much faster; Qwen spent far more tokens.

Wësche

51,527 Aufrufe • vor 2 Monaten

MiniMax M3.1 Flash vs DeepSeek at Tesla Roadster 🚔 MiniMax M3.1 Flash re-created this viral Tesla Roadster police chase scene for $2.34, while DeepSeek V4.1 Flash's version cost $3.07 Run AI models locally or via API ->
0:18

MiniMax M3.1 Flash vs DeepSeek at Tesla Roadster 🚔 MiniMax M3.1 Flash re-created this viral Tesla Roadster police chase scene for $2.34, while DeepSeek V4.1 Flash's version cost $3.07 Run AI models locally or via API ->

atomic.chat

44,413 Aufrufe • vor 27 Tagen

Ornith-1.0 35B - MLX Device - 128gb m5 max mbp Harness - PocketAI Hub 4 bit - 11,504 tokens, 95.98 tok/s, 21.20 GB peak RAM 8 bit - 13,128 tokens, 64.96 tok/s, 38.92 GB peak RAM BF16 - 12,491 tokens, 47.72 tok/s, 70.59 GB peak RAM prompt and link to model below
0:15

Ornith-1.0 35B - MLX Device - 128gb m5 max mbp Harness - PocketAI Hub 4 bit - 11,504 tokens, 95.98 tok/s, 21.20 GB peak RAM 8 bit - 13,128 tokens, 64.96 tok/s, 38.92 GB peak RAM BF16 - 12,491 tokens, 47.72 tok/s, 70.59 GB peak RAM prompt and link to model below

Trevor Wood

11,765 Aufrufe • vor 3 Monaten

🏁 BENCHMARKED: webAI's TwIL-8B, an 8.8B open-weight reasoning model, just went 69/69 on my tests running on my 64GB M5 Pro Mac mini. The 4-bit MLX build runs at 53.8 tok/s from a 4.9 GB file. Full bf16 with the whole 128K context fits too, peaking at 40.5 GB on a 64GB machine. Same scores on both builds, thinking on: ✅ Math 20/20 ✅ Code 10/10 ✅ Instruction following 10/10 ✅ Harder math 8/8 ✅ Harder code 5/5 ✅ Tool calls 5/5 ✅ Hidden fact at 32K and 64K: 8/8 ✅ Hidden fact at full 128K: 3/3 (bf16) 4-bit speed by context: - 512: 53.8 tok/s - 32K: 26.6 tok/s - 128K: 9.2 tok/s at 27.9 GB peak bf16 speed by context: - 512: 16.7 tok/s - 32K: 12.6 tok/s - 128K: 6.6 tok/s David Stout called it "small enough to run on your own hardware." I checked and it runs at full precision with the whole 128K context on a desk I own.
0:16

🏁 BENCHMARKED: webAI's TwIL-8B, an 8.8B open-weight reasoning model, just went 69/69 on my tests running on my 64GB M5 Pro Mac mini. The 4-bit MLX build runs at 53.8 tok/s from a 4.9 GB file. Full bf16 with the whole 128K context fits too, peaking at 40.5 GB on a 64GB machine. Same scores on both builds, thinking on: ✅ Math 20/20 ✅ Code 10/10 ✅ Instruction following 10/10 ✅ Harder math 8/8 ✅ Harder code 5/5 ✅ Tool calls 5/5 ✅ Hidden fact at 32K and 64K: 8/8 ✅ Hidden fact at full 128K: 3/3 (bf16) 4-bit speed by context: - 512: 53.8 tok/s - 32K: 26.6 tok/s - 128K: 9.2 tok/s at 27.9 GB peak bf16 speed by context: - 512: 16.7 tok/s - 32K: 12.6 tok/s - 128K: 6.6 tok/s David Stout called it "small enough to run on your own hardware." I checked and it runs at full precision with the whole 128K context on a desk I own.

Pratik Gandhi

11,711 Aufrufe • vor 10 Tagen