Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Guess the tok/s

0xSero's profile picture

0xSero

68,889 subscribers

17,255 Aufrufe • vor 3 Tagen •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

36 Kommentare

Profilbild von Sufyan
Sufyanvor 3 Tagen

500

Profilbild von Miyuru
Miyuruvor 3 Tagen

locking in 900. now tell me the model and how many cards are coughing up smoke behind it

Profilbild von Meet Limbani
Meet Limbanivor 3 Tagen

i know that scrolling speed in PI, it's around 350

Profilbild von Gil
Gilvor 3 Tagen

750

Profilbild von Neo
Neovor 3 Tagen

420

Profilbild von 0xSero
0xSerovor 3 Tagen

420.69

Profilbild von Decatalyst 🌱
Decatalyst 🌱vor 3 Tagen

OH MY TOMATOES I FEEL LIKE A CAVEMAN USING OPUS 5 NOW 😭

Profilbild von ruleryak
ruleryakvor 3 Tagen

went from 24k output at 7s to 44k output at 34s so end to end throughput including tool calls, prefill, and decode is around 750 tps there. So the actual decode is probably 1k+

Profilbild von Maciej
Maciejvor 3 Tagen

No tool call streaming? They spend bilions on hardware and lack money for API software.

Profilbild von Valentin Yanakiev
Valentin Yanakievvor 3 Tagen

384

Profilbild von D
Dvor 3 Tagen

450

Profilbild von pythongiant
pythongiantvor 3 Tagen

a majillion

Profilbild von PaulNL
PaulNLvor 3 Tagen

400ish ?

Profilbild von Mike Reese
Mike Reesevor 3 Tagen

How does cerebras compare to your local setup with deepseek 4.1 flash in terms of tok/s?

Profilbild von AArchimedes64
AArchimedes64vor 3 Tagen

1024

Profilbild von cherki
cherkivor 3 Tagen

How?

Profilbild von Arman
Armanvor 3 Tagen

1337

Profilbild von Fraser Price
Fraser Pricevor 3 Tagen

Bout tree fiddy

Profilbild von Crispybits || Parroty account
Crispybits || Parroty accountvor 3 Tagen

3 - you've sped up the AI writing sections of the video 1000x 😹

Profilbild von Redmix
Redmixvor 3 Tagen

~1850

Profilbild von Wayne
Waynevor 3 Tagen

I bet that's at least 1t/s

Profilbild von Serhii Y.
Serhii Y.vor 3 Tagen

you're typing 1-2tts, model 100+

Profilbild von mo
movor 3 Tagen

320

Profilbild von Kutluk
Kutlukvor 3 Tagen

~300-400?

Profilbild von Jason Frisch
Jason Frischvor 3 Tagen

What do I need to run GLM-5.3-flash? Can a Mac Studio manage it for regular coding tasks?

Profilbild von Loco Legend
Loco Legendvor 3 Tagen

170-250

Profilbild von Sektur Toilet
Sektur Toiletvor 3 Tagen

3-400?

Profilbild von Jeff Steve
Jeff Stevevor 3 Tagen

700

Profilbild von Simao Soares 🇪🇺🇵🇹🇳🇱
Simao Soares 🇪🇺🇵🇹🇳🇱vor 3 Tagen

I've seen a tok/s, is there a metric for token quality? I'm honestly curious. So many tools to reduce token usage like context compression, context understanding, prompt engineering for summarised high quality replies, MTP and non-MTP...

Profilbild von saltyclaw707
saltyclaw707vor 3 Tagen

300 ish

Profilbild von vveerrgg
vveerrggvor 3 Tagen

280 T/Sec. … how are you configuring your cards … that’s the question I have

Profilbild von Petey struggles
Petey strugglesvor 3 Tagen

1500

Profilbild von Matt
Mattvor 3 Tagen

700+

Profilbild von Asif Sheikh 💸
Asif Sheikh 💸vor 3 Tagen

~1,850 tokens/s you're welcome.

Profilbild von byovd
byovdvor 3 Tagen

300

Profilbild von Kachow Towmater
Kachow Towmatervor 3 Tagen

280

Ähnliche Videos

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.
1:10

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

0xSero

30,601 Aufrufe • vor 22 Tagen

Kimi-K2.6 running locally 110+ tok/s
0:42

Kimi-K2.6 running locally 110+ tok/s

0xSero

24,640 Aufrufe • vor 3 Monaten

Nemotron-3-Ultra running on 4x 6000s edits my latest demo video.. - 75 tok/s decode - 8x concurrency - 256k context - 899 tok/s prefill - 20k tok/s prefill cache - NVFP4 Setting it up to be my Hermes driver. It's good enough at most things and doesn't talk like a moron.
1:26

Nemotron-3-Ultra running on 4x 6000s edits my latest demo video.. - 75 tok/s decode - 8x concurrency - 256k context - 899 tok/s prefill - 20k tok/s prefill cache - NVFP4 Setting it up to be my Hermes driver. It's good enough at most things and doesn't talk like a moron.

0xSero

15,674 Aufrufe • vor 3 Monaten

Opus at home, at 200+ tok/s I love ZAI
0:33

Opus at home, at 200+ tok/s I love ZAI

0xSero

201,224 Aufrufe • vor 23 Tagen

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:
1:00

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:

0xSero

74,772 Aufrufe • vor 5 Monaten

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.
0:28

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

0xSero

157,562 Aufrufe • vor 2 Monaten

Minimax-M3 running on 4x RTX Pro 6000s - 800k context - 4x concurrency at 250k - 70-120 tok/s - 2000 tok/s prefill no cache - 376gb vram - mxfp4 It's working on improving the audio on one of my videos, it's actually doing a good job in researching solutions. Good model
0:33

Minimax-M3 running on 4x RTX Pro 6000s - 800k context - 4x concurrency at 250k - 70-120 tok/s - 2000 tok/s prefill no cache - 376gb vram - mxfp4 It's working on improving the audio on one of my videos, it's actually doing a good job in researching solutions. Good model

0xSero

24,634 Aufrufe • vor 3 Monaten

MiniMax-M3 running locally 100+ tok/s Good questions not often that models ask good questions.
1:23

MiniMax-M3 running locally 100+ tok/s Good questions not often that models ask good questions.

0xSero

32,728 Aufrufe • vor 3 Monaten

I am blown away by MiMo-V2.5 It's so steerable, I didn't like MiMo-V2 This is self hosted btw, 125 tok/s across 2 sessions 4k prefill tok/s It's updating my website + editing a longer video I will be posting in a few minutes.
0:25

I am blown away by MiMo-V2.5 It's so steerable, I didn't like MiMo-V2 This is self hosted btw, 125 tok/s across 2 sessions 4k prefill tok/s It's updating my website + editing a longer video I will be posting in a few minutes.

0xSero

34,494 Aufrufe • vor 4 Monaten

Okay my hype levels are out of this world rn. I quantised the engram to fp4 and now don't need nvme offload, decode is 80% faster, at 260 tok/s now
0:53

Okay my hype levels are out of this world rn. I quantised the engram to fp4 and now don't need nvme offload, decode is 80% faster, at 260 tok/s now

0xSero

40,748 Aufrufe • vor 8 Tagen

Curious about ollama kimi-k2.6:cloud speed? 3 test runs: > 77.9 tok/s, TTFT 979ms > 114.3 tok/s, TTFT 788ms > 86.3 tok/s, TTFT 1117ms For comparison, OpenRouter stats: > Parasail: 14 tok/s > Moonshot AI: 27 tok/s > NovitaAI: 27 tok/s > Cloudflare: 71 tok/s Obvious caveat: cloud speeds fluctuate with load. Just sharing numbers for the curious.
0:58

Curious about ollama kimi-k2.6:cloud speed? 3 test runs: > 77.9 tok/s, TTFT 979ms > 114.3 tok/s, TTFT 788ms > 86.3 tok/s, TTFT 1117ms For comparison, OpenRouter stats: > Parasail: 14 tok/s > Moonshot AI: 27 tok/s > NovitaAI: 27 tok/s > Cloudflare: 71 tok/s Obvious caveat: cloud speeds fluctuate with load. Just sharing numbers for the curious.

stevibe

35,129 Aufrufe • vor 5 Monaten

Gemma-4-31B builds a website from a wireframe I sketched out in Excalidraw Not perfect but really really fast with Spec decoding 150 tok/s+ and it's good at following instructions Free workhorse. Use speculative decoding, it's only 0.7GB of vram for a 2x speed up! vllm-studio
3:46

Gemma-4-31B builds a website from a wireframe I sketched out in Excalidraw Not perfect but really really fast with Spec decoding 150 tok/s+ and it's good at following instructions Free workhorse. Use speculative decoding, it's only 0.7GB of vram for a 2x speed up! vllm-studio

0xSero

21,768 Aufrufe • vor 4 Monaten

The RTX 3090 is a 5-year-old GPU and it still runs a 27B model at 20 tok/s I tested Qwen3.5:27b across 3 generations of NVIDIA: 5090 → ~60 tok/s 4090 → ~40 tok/s 3090 → ~20 tok/s Perfectly linear scaling. Double the generation, double the speed.
0:57

The RTX 3090 is a 5-year-old GPU and it still runs a 27B model at 20 tok/s I tested Qwen3.5:27b across 3 generations of NVIDIA: 5090 → ~60 tok/s 4090 → ~40 tok/s 3090 → ~20 tok/s Perfectly linear scaling. Double the generation, double the speed.

stevibe

142,493 Aufrufe • vor 6 Monaten

Guess who got GLM-5.2 at home? You can't deny I'm elite at inference.
0:31

Guess who got GLM-5.2 at home? You can't deny I'm elite at inference.

0xSero

27,083 Aufrufe • vor 3 Monaten

DGX Spark hit 228ms time-to-first-token on Qwen3.6 35B-A3B. That is 4x faster than an RTX 3090 and 45% ahead of the RTX 5090 Raw throughput still belongs to the 5090 at 160 tok/s. Full numbers, same Ollama backend, Q4_K_M across all four: RTX 3090: 49.78 tok/s, 852ms RTX 4090: 118.93 tok/s, 686ms RTX 5090: 160.37 tok/s, 409ms DGX Spark: 59.98 tok/s, 228ms
1:24

DGX Spark hit 228ms time-to-first-token on Qwen3.6 35B-A3B. That is 4x faster than an RTX 3090 and 45% ahead of the RTX 5090 Raw throughput still belongs to the 5090 at 160 tok/s. Full numbers, same Ollama backend, Q4_K_M across all four: RTX 3090: 49.78 tok/s, 852ms RTX 4090: 118.93 tok/s, 686ms RTX 5090: 160.37 tok/s, 409ms DGX Spark: 59.98 tok/s, 228ms

NO1ennn

22,497 Aufrufe • vor 2 Monaten

Finally, full precision MiniMax-M2.7 running at home. 100 tokens/s decode 5050 tokens/s prefill
0:36

Finally, full precision MiniMax-M2.7 running at home. 100 tokens/s decode 5050 tokens/s prefill

0xSero

72,465 Aufrufe • vor 5 Monaten

Gemma4 12B with Unsloth's Quant on DGX Spark Quants: - UD_Q4_K_XL - UD_Q5_K_XL - UD_Q6_K_XL - UD_Q8_K_XL Summary: - Q4: 25.21 tok/s, TTFT 168ms - Q5: 21.7 tok/s, TTFT 182ms - Q6: 17.68 tok/s, TTFT 193.95ms - Q8: 15.22 tok/s, TTFT 221ms
1:03

Gemma4 12B with Unsloth's Quant on DGX Spark Quants: - UD_Q4_K_XL - UD_Q5_K_XL - UD_Q6_K_XL - UD_Q8_K_XL Summary: - Q4: 25.21 tok/s, TTFT 168ms - Q5: 21.7 tok/s, TTFT 182ms - Q6: 17.68 tok/s, TTFT 193.95ms - Q8: 15.22 tok/s, TTFT 221ms

stevibe

18,639 Aufrufe • vor 3 Monaten

Almost ready, I guess I will finally work on the onboarding experience. I have been using vllm-studio for work all day, this is so good. I legit haven't noticed a single issue with the model, the outputs or even the app's usability. You will have your coding agent at home.
2:15

Almost ready, I guess I will finally work on the onboarding experience. I have been using vllm-studio for work all day, this is so good. I legit haven't noticed a single issue with the model, the outputs or even the app's usability. You will have your coding agent at home.

0xSero

14,320 Aufrufe • vor 4 Monaten

This is the FASTER MODEL I've EVER Ran LOCALLY 🚀 laguna on Laguna S 2.1 · 3090 Quad INT4 • 1 parallel coding agents • Total: 3,405 tokens in 14s • Aggregate: 256.0 tok/s peak · 249.9 tok/s sustained • Per-stream: 256.0 high / 256.0 low / 256.0 avg tok/s • TTFT 0.12s avg
1:58

This is the FASTER MODEL I've EVER Ran LOCALLY 🚀 laguna on Laguna S 2.1 · 3090 Quad INT4 • 1 parallel coding agents • Total: 3,405 tokens in 14s • Aggregate: 256.0 tok/s peak · 249.9 tok/s sustained • Per-stream: 256.0 high / 256.0 low / 256.0 avg tok/s • TTFT 0.12s avg

Tech2Wild

16,404 Aufrufe • vor 1 Monat