正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Guess the tok/s

0xSero

68,889 subscribers

17,255 次观看 • 3 天前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

36 条评论

Sufyan 的头像

Sufyan3 天前

500

Miyuru 的头像

Miyuru3 天前

locking in 900. now tell me the model and how many cards are coughing up smoke behind it

Meet Limbani 的头像

Meet Limbani3 天前

i know that scrolling speed in PI, it's around 350

Gil 的头像

Gil3 天前

750

Neo 的头像

Neo3 天前

420

0xSero 的头像

0xSero3 天前

420.69

Decatalyst 🌱 的头像

Decatalyst 🌱3 天前

OH MY TOMATOES I FEEL LIKE A CAVEMAN USING OPUS 5 NOW 😭

ruleryak 的头像

ruleryak3 天前

went from 24k output at 7s to 44k output at 34s so end to end throughput including tool calls, prefill, and decode is around 750 tps there. So the actual decode is probably 1k+

Maciej 的头像

Maciej3 天前

No tool call streaming? They spend bilions on hardware and lack money for API software.

Valentin Yanakiev 的头像

Valentin Yanakiev3 天前

384

D 的头像

D3 天前

450

pythongiant 的头像

pythongiant3 天前

a majillion

PaulNL 的头像

PaulNL3 天前

400ish ?

Mike Reese 的头像

Mike Reese3 天前

How does cerebras compare to your local setup with deepseek 4.1 flash in terms of tok/s?

AArchimedes64 的头像

AArchimedes643 天前

1024

cherki 的头像

cherki3 天前

How?

Arman 的头像

Arman3 天前

1337

Fraser Price 的头像

Fraser Price3 天前

Bout tree fiddy

Crispybits || Parroty account 的头像

Crispybits || Parroty account3 天前

3 - you've sped up the AI writing sections of the video 1000x 😹

Redmix 的头像

Redmix3 天前

~1850

Wayne 的头像

Wayne3 天前

I bet that's at least 1t/s

Serhii Y. 的头像

Serhii Y.3 天前

you're typing 1-2tts, model 100+

mo 的头像

mo3 天前

320

Kutluk 的头像

Kutluk3 天前

~300-400?

Jason Frisch 的头像

Jason Frisch3 天前

What do I need to run GLM-5.3-flash? Can a Mac Studio manage it for regular coding tasks?

Loco Legend 的头像

Loco Legend3 天前

170-250

Sektur Toilet 的头像

Sektur Toilet3 天前

3-400?

Jeff Steve 的头像

Jeff Steve3 天前

700

Simao Soares 🇪🇺🇵🇹🇳🇱 的头像

Simao Soares 🇪🇺🇵🇹🇳🇱3 天前

I've seen a tok/s, is there a metric for token quality? I'm honestly curious. So many tools to reduce token usage like context compression, context understanding, prompt engineering for summarised high quality replies, MTP and non-MTP...

saltyclaw707 的头像

saltyclaw7073 天前

300 ish

vveerrgg 的头像

vveerrgg3 天前

280 T/Sec. … how are you configuring your cards … that’s the question I have

Petey struggles 的头像

Petey struggles3 天前

1500

Matt 的头像

Matt3 天前

700+

Asif Sheikh 💸 的头像

Asif Sheikh 💸3 天前

~1,850 tokens/s you're welcome.

byovd 的头像

byovd3 天前

300

Kachow Towmater 的头像

Kachow Towmater3 天前

280

相关视频

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

0xSero

30,601 次观看 • 22 天前

Kimi-K2.6 running locally 110+ tok/s

Kimi-K2.6 running locally 110+ tok/s

0xSero

24,640 次观看 • 3 个月前

Nemotron-3-Ultra running on 4x 6000s edits my latest demo video.. - 75 tok/s decode - 8x concurrency - 256k context - 899 tok/s prefill - 20k tok/s prefill cache - NVFP4 Setting it up to be my Hermes driver. It's good enough at most things and doesn't talk like a moron.

Nemotron-3-Ultra running on 4x 6000s edits my latest demo video.. - 75 tok/s decode - 8x concurrency - 256k context - 899 tok/s prefill - 20k tok/s prefill cache - NVFP4 Setting it up to be my Hermes driver. It's good enough at most things and doesn't talk like a moron.

0xSero

15,674 次观看 • 3 个月前

Opus at home, at 200+ tok/s I love ZAI

Opus at home, at 200+ tok/s I love ZAI

0xSero

201,224 次观看 • 23 天前

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:

0xSero

74,772 次观看 • 5 个月前

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

0xSero

157,562 次观看 • 2 个月前

Minimax-M3 running on 4x RTX Pro 6000s - 800k context - 4x concurrency at 250k - 70-120 tok/s - 2000 tok/s prefill no cache - 376gb vram - mxfp4 It's working on improving the audio on one of my videos, it's actually doing a good job in researching solutions. Good model

Minimax-M3 running on 4x RTX Pro 6000s - 800k context - 4x concurrency at 250k - 70-120 tok/s - 2000 tok/s prefill no cache - 376gb vram - mxfp4 It's working on improving the audio on one of my videos, it's actually doing a good job in researching solutions. Good model

0xSero

24,634 次观看 • 3 个月前

MiniMax-M3 running locally 100+ tok/s Good questions not often that models ask good questions.

MiniMax-M3 running locally 100+ tok/s Good questions not often that models ask good questions.

0xSero

32,728 次观看 • 3 个月前

I am blown away by MiMo-V2.5 It's so steerable, I didn't like MiMo-V2 This is self hosted btw, 125 tok/s across 2 sessions 4k prefill tok/s It's updating my website + editing a longer video I will be posting in a few minutes.

I am blown away by MiMo-V2.5 It's so steerable, I didn't like MiMo-V2 This is self hosted btw, 125 tok/s across 2 sessions 4k prefill tok/s It's updating my website + editing a longer video I will be posting in a few minutes.

0xSero

34,494 次观看 • 4 个月前

Okay my hype levels are out of this world rn. I quantised the engram to fp4 and now don't need nvme offload, decode is 80% faster, at 260 tok/s now

Okay my hype levels are out of this world rn. I quantised the engram to fp4 and now don't need nvme offload, decode is 80% faster, at 260 tok/s now

0xSero

40,748 次观看 • 8 天前

Curious about ollama kimi-k2.6:cloud speed? 3 test runs: > 77.9 tok/s, TTFT 979ms > 114.3 tok/s, TTFT 788ms > 86.3 tok/s, TTFT 1117ms For comparison, OpenRouter stats: > Parasail: 14 tok/s > Moonshot AI: 27 tok/s > NovitaAI: 27 tok/s > Cloudflare: 71 tok/s Obvious caveat: cloud speeds fluctuate with load. Just sharing numbers for the curious.

Curious about ollama kimi-k2.6:cloud speed? 3 test runs: > 77.9 tok/s, TTFT 979ms > 114.3 tok/s, TTFT 788ms > 86.3 tok/s, TTFT 1117ms For comparison, OpenRouter stats: > Parasail: 14 tok/s > Moonshot AI: 27 tok/s > NovitaAI: 27 tok/s > Cloudflare: 71 tok/s Obvious caveat: cloud speeds fluctuate with load. Just sharing numbers for the curious.

stevibe

35,129 次观看 • 5 个月前

Gemma-4-31B builds a website from a wireframe I sketched out in Excalidraw Not perfect but really really fast with Spec decoding 150 tok/s+ and it's good at following instructions Free workhorse. Use speculative decoding, it's only 0.7GB of vram for a 2x speed up! vllm-studio

Gemma-4-31B builds a website from a wireframe I sketched out in Excalidraw Not perfect but really really fast with Spec decoding 150 tok/s+ and it's good at following instructions Free workhorse. Use speculative decoding, it's only 0.7GB of vram for a 2x speed up! vllm-studio

0xSero

21,768 次观看 • 4 个月前

The RTX 3090 is a 5-year-old GPU and it still runs a 27B model at 20 tok/s I tested Qwen3.5:27b across 3 generations of NVIDIA: 5090 → ~60 tok/s 4090 → ~40 tok/s 3090 → ~20 tok/s Perfectly linear scaling. Double the generation, double the speed.

The RTX 3090 is a 5-year-old GPU and it still runs a 27B model at 20 tok/s I tested Qwen3.5:27b across 3 generations of NVIDIA: 5090 → ~60 tok/s 4090 → ~40 tok/s 3090 → ~20 tok/s Perfectly linear scaling. Double the generation, double the speed.

stevibe

142,493 次观看 • 6 个月前

Guess who got GLM-5.2 at home? You can't deny I'm elite at inference.

Guess who got GLM-5.2 at home? You can't deny I'm elite at inference.

0xSero

27,083 次观看 • 3 个月前

DGX Spark hit 228ms time-to-first-token on Qwen3.6 35B-A3B. That is 4x faster than an RTX 3090 and 45% ahead of the RTX 5090 Raw throughput still belongs to the 5090 at 160 tok/s. Full numbers, same Ollama backend, Q4_K_M across all four: RTX 3090: 49.78 tok/s, 852ms RTX 4090: 118.93 tok/s, 686ms RTX 5090: 160.37 tok/s, 409ms DGX Spark: 59.98 tok/s, 228ms

DGX Spark hit 228ms time-to-first-token on Qwen3.6 35B-A3B. That is 4x faster than an RTX 3090 and 45% ahead of the RTX 5090 Raw throughput still belongs to the 5090 at 160 tok/s. Full numbers, same Ollama backend, Q4_K_M across all four: RTX 3090: 49.78 tok/s, 852ms RTX 4090: 118.93 tok/s, 686ms RTX 5090: 160.37 tok/s, 409ms DGX Spark: 59.98 tok/s, 228ms

NO1ennn

22,497 次观看 • 2 个月前

Finally, full precision MiniMax-M2.7 running at home. 100 tokens/s decode 5050 tokens/s prefill

Finally, full precision MiniMax-M2.7 running at home. 100 tokens/s decode 5050 tokens/s prefill

0xSero

72,465 次观看 • 5 个月前

Gemma4 12B with Unsloth's Quant on DGX Spark Quants: - UD_Q4_K_XL - UD_Q5_K_XL - UD_Q6_K_XL - UD_Q8_K_XL Summary: - Q4: 25.21 tok/s, TTFT 168ms - Q5: 21.7 tok/s, TTFT 182ms - Q6: 17.68 tok/s, TTFT 193.95ms - Q8: 15.22 tok/s, TTFT 221ms

Gemma4 12B with Unsloth's Quant on DGX Spark Quants: - UD_Q4_K_XL - UD_Q5_K_XL - UD_Q6_K_XL - UD_Q8_K_XL Summary: - Q4: 25.21 tok/s, TTFT 168ms - Q5: 21.7 tok/s, TTFT 182ms - Q6: 17.68 tok/s, TTFT 193.95ms - Q8: 15.22 tok/s, TTFT 221ms

stevibe

18,639 次观看 • 3 个月前

Almost ready, I guess I will finally work on the onboarding experience. I have been using vllm-studio for work all day, this is so good. I legit haven't noticed a single issue with the model, the outputs or even the app's usability. You will have your coding agent at home.

Almost ready, I guess I will finally work on the onboarding experience. I have been using vllm-studio for work all day, this is so good. I legit haven't noticed a single issue with the model, the outputs or even the app's usability. You will have your coding agent at home.

0xSero

14,320 次观看 • 4 个月前

This is the FASTER MODEL I've EVER Ran LOCALLY 🚀 laguna on Laguna S 2.1 · 3090 Quad INT4 • 1 parallel coding agents • Total: 3,405 tokens in 14s • Aggregate: 256.0 tok/s peak · 249.9 tok/s sustained • Per-stream: 256.0 high / 256.0 low / 256.0 avg tok/s • TTFT 0.12s avg

This is the FASTER MODEL I've EVER Ran LOCALLY 🚀 laguna on Laguna S 2.1 · 3090 Quad INT4 • 1 parallel coding agents • Total: 3,405 tokens in 14s • Aggregate: 256.0 tok/s peak · 249.9 tok/s sustained • Per-stream: 256.0 high / 256.0 low / 256.0 avg tok/s • TTFT 0.12s avg

Tech2Wild

16,404 次观看 • 1 个月前