Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Guess the tok/s

0xSero's profile picture

0xSero

68,889 subscribers

17,255 görüntüleme • 3 gün önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

36 Yorum

Sufyan profil fotoğrafı
Sufyan3 gün önce

500

Miyuru profil fotoğrafı
Miyuru3 gün önce

locking in 900. now tell me the model and how many cards are coughing up smoke behind it

Meet Limbani profil fotoğrafı
Meet Limbani3 gün önce

i know that scrolling speed in PI, it's around 350

Gil profil fotoğrafı
Gil3 gün önce

750

Neo profil fotoğrafı
Neo3 gün önce

420

0xSero profil fotoğrafı
0xSero3 gün önce

420.69

Decatalyst 🌱 profil fotoğrafı
Decatalyst 🌱3 gün önce

OH MY TOMATOES I FEEL LIKE A CAVEMAN USING OPUS 5 NOW 😭

ruleryak profil fotoğrafı
ruleryak3 gün önce

went from 24k output at 7s to 44k output at 34s so end to end throughput including tool calls, prefill, and decode is around 750 tps there. So the actual decode is probably 1k+

Maciej profil fotoğrafı
Maciej3 gün önce

No tool call streaming? They spend bilions on hardware and lack money for API software.

Valentin Yanakiev profil fotoğrafı
Valentin Yanakiev3 gün önce

384

D profil fotoğrafı
D3 gün önce

450

pythongiant profil fotoğrafı
pythongiant3 gün önce

a majillion

PaulNL profil fotoğrafı
PaulNL3 gün önce

400ish ?

Mike Reese profil fotoğrafı
Mike Reese3 gün önce

How does cerebras compare to your local setup with deepseek 4.1 flash in terms of tok/s?

AArchimedes64 profil fotoğrafı
AArchimedes643 gün önce

1024

cherki profil fotoğrafı
cherki3 gün önce

How?

Arman profil fotoğrafı
Arman3 gün önce

1337

Fraser Price profil fotoğrafı
Fraser Price3 gün önce

Bout tree fiddy

Crispybits || Parroty account profil fotoğrafı
Crispybits || Parroty account3 gün önce

3 - you've sped up the AI writing sections of the video 1000x 😹

Redmix profil fotoğrafı
Redmix3 gün önce

~1850

Wayne profil fotoğrafı
Wayne3 gün önce

I bet that's at least 1t/s

Serhii Y. profil fotoğrafı
Serhii Y.3 gün önce

you're typing 1-2tts, model 100+

mo profil fotoğrafı
mo3 gün önce

320

Kutluk profil fotoğrafı
Kutluk3 gün önce

~300-400?

Jason Frisch profil fotoğrafı
Jason Frisch3 gün önce

What do I need to run GLM-5.3-flash? Can a Mac Studio manage it for regular coding tasks?

Loco Legend profil fotoğrafı
Loco Legend3 gün önce

170-250

Sektur Toilet profil fotoğrafı
Sektur Toilet3 gün önce

3-400?

Jeff Steve profil fotoğrafı
Jeff Steve3 gün önce

700

Simao Soares 🇪🇺🇵🇹🇳🇱 profil fotoğrafı
Simao Soares 🇪🇺🇵🇹🇳🇱3 gün önce

I've seen a tok/s, is there a metric for token quality? I'm honestly curious. So many tools to reduce token usage like context compression, context understanding, prompt engineering for summarised high quality replies, MTP and non-MTP...

saltyclaw707 profil fotoğrafı
saltyclaw7073 gün önce

300 ish

vveerrgg profil fotoğrafı
vveerrgg3 gün önce

280 T/Sec. … how are you configuring your cards … that’s the question I have

Petey struggles profil fotoğrafı
Petey struggles3 gün önce

1500

Matt profil fotoğrafı
Matt3 gün önce

700+

Asif Sheikh 💸 profil fotoğrafı
Asif Sheikh 💸3 gün önce

~1,850 tokens/s you're welcome.

byovd profil fotoğrafı
byovd3 gün önce

300

Kachow Towmater profil fotoğrafı
Kachow Towmater3 gün önce

280

Benzer Videolar

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.
1:10

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

0xSero

30,601 görüntüleme • 22 gün önce

Kimi-K2.6 running locally 110+ tok/s
0:42

Kimi-K2.6 running locally 110+ tok/s

0xSero

24,640 görüntüleme • 3 ay önce

Nemotron-3-Ultra running on 4x 6000s edits my latest demo video.. - 75 tok/s decode - 8x concurrency - 256k context - 899 tok/s prefill - 20k tok/s prefill cache - NVFP4 Setting it up to be my Hermes driver. It's good enough at most things and doesn't talk like a moron.
1:26

Nemotron-3-Ultra running on 4x 6000s edits my latest demo video.. - 75 tok/s decode - 8x concurrency - 256k context - 899 tok/s prefill - 20k tok/s prefill cache - NVFP4 Setting it up to be my Hermes driver. It's good enough at most things and doesn't talk like a moron.

0xSero

15,674 görüntüleme • 3 ay önce

Opus at home, at 200+ tok/s I love ZAI
0:33

Opus at home, at 200+ tok/s I love ZAI

0xSero

201,224 görüntüleme • 23 gün önce

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:
1:00

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:

0xSero

74,772 görüntüleme • 5 ay önce

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.
0:28

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

0xSero

157,562 görüntüleme • 2 ay önce

Minimax-M3 running on 4x RTX Pro 6000s - 800k context - 4x concurrency at 250k - 70-120 tok/s - 2000 tok/s prefill no cache - 376gb vram - mxfp4 It's working on improving the audio on one of my videos, it's actually doing a good job in researching solutions. Good model
0:33

Minimax-M3 running on 4x RTX Pro 6000s - 800k context - 4x concurrency at 250k - 70-120 tok/s - 2000 tok/s prefill no cache - 376gb vram - mxfp4 It's working on improving the audio on one of my videos, it's actually doing a good job in researching solutions. Good model

0xSero

24,634 görüntüleme • 3 ay önce

MiniMax-M3 running locally 100+ tok/s Good questions not often that models ask good questions.
1:23

MiniMax-M3 running locally 100+ tok/s Good questions not often that models ask good questions.

0xSero

32,728 görüntüleme • 3 ay önce

I am blown away by MiMo-V2.5 It's so steerable, I didn't like MiMo-V2 This is self hosted btw, 125 tok/s across 2 sessions 4k prefill tok/s It's updating my website + editing a longer video I will be posting in a few minutes.
0:25

I am blown away by MiMo-V2.5 It's so steerable, I didn't like MiMo-V2 This is self hosted btw, 125 tok/s across 2 sessions 4k prefill tok/s It's updating my website + editing a longer video I will be posting in a few minutes.

0xSero

34,494 görüntüleme • 4 ay önce

Okay my hype levels are out of this world rn. I quantised the engram to fp4 and now don't need nvme offload, decode is 80% faster, at 260 tok/s now
0:53

Okay my hype levels are out of this world rn. I quantised the engram to fp4 and now don't need nvme offload, decode is 80% faster, at 260 tok/s now

0xSero

40,748 görüntüleme • 8 gün önce

Curious about ollama kimi-k2.6:cloud speed? 3 test runs: > 77.9 tok/s, TTFT 979ms > 114.3 tok/s, TTFT 788ms > 86.3 tok/s, TTFT 1117ms For comparison, OpenRouter stats: > Parasail: 14 tok/s > Moonshot AI: 27 tok/s > NovitaAI: 27 tok/s > Cloudflare: 71 tok/s Obvious caveat: cloud speeds fluctuate with load. Just sharing numbers for the curious.
0:58

Curious about ollama kimi-k2.6:cloud speed? 3 test runs: > 77.9 tok/s, TTFT 979ms > 114.3 tok/s, TTFT 788ms > 86.3 tok/s, TTFT 1117ms For comparison, OpenRouter stats: > Parasail: 14 tok/s > Moonshot AI: 27 tok/s > NovitaAI: 27 tok/s > Cloudflare: 71 tok/s Obvious caveat: cloud speeds fluctuate with load. Just sharing numbers for the curious.

stevibe

35,129 görüntüleme • 5 ay önce

Gemma-4-31B builds a website from a wireframe I sketched out in Excalidraw Not perfect but really really fast with Spec decoding 150 tok/s+ and it's good at following instructions Free workhorse. Use speculative decoding, it's only 0.7GB of vram for a 2x speed up! vllm-studio
3:46

Gemma-4-31B builds a website from a wireframe I sketched out in Excalidraw Not perfect but really really fast with Spec decoding 150 tok/s+ and it's good at following instructions Free workhorse. Use speculative decoding, it's only 0.7GB of vram for a 2x speed up! vllm-studio

0xSero

21,768 görüntüleme • 4 ay önce

The RTX 3090 is a 5-year-old GPU and it still runs a 27B model at 20 tok/s I tested Qwen3.5:27b across 3 generations of NVIDIA: 5090 → ~60 tok/s 4090 → ~40 tok/s 3090 → ~20 tok/s Perfectly linear scaling. Double the generation, double the speed.
0:57

The RTX 3090 is a 5-year-old GPU and it still runs a 27B model at 20 tok/s I tested Qwen3.5:27b across 3 generations of NVIDIA: 5090 → ~60 tok/s 4090 → ~40 tok/s 3090 → ~20 tok/s Perfectly linear scaling. Double the generation, double the speed.

stevibe

142,493 görüntüleme • 6 ay önce

Guess who got GLM-5.2 at home? You can't deny I'm elite at inference.
0:31

Guess who got GLM-5.2 at home? You can't deny I'm elite at inference.

0xSero

27,083 görüntüleme • 3 ay önce

DGX Spark hit 228ms time-to-first-token on Qwen3.6 35B-A3B. That is 4x faster than an RTX 3090 and 45% ahead of the RTX 5090 Raw throughput still belongs to the 5090 at 160 tok/s. Full numbers, same Ollama backend, Q4_K_M across all four: RTX 3090: 49.78 tok/s, 852ms RTX 4090: 118.93 tok/s, 686ms RTX 5090: 160.37 tok/s, 409ms DGX Spark: 59.98 tok/s, 228ms
1:24

DGX Spark hit 228ms time-to-first-token on Qwen3.6 35B-A3B. That is 4x faster than an RTX 3090 and 45% ahead of the RTX 5090 Raw throughput still belongs to the 5090 at 160 tok/s. Full numbers, same Ollama backend, Q4_K_M across all four: RTX 3090: 49.78 tok/s, 852ms RTX 4090: 118.93 tok/s, 686ms RTX 5090: 160.37 tok/s, 409ms DGX Spark: 59.98 tok/s, 228ms

NO1ennn

22,497 görüntüleme • 2 ay önce

Finally, full precision MiniMax-M2.7 running at home. 100 tokens/s decode 5050 tokens/s prefill
0:36

Finally, full precision MiniMax-M2.7 running at home. 100 tokens/s decode 5050 tokens/s prefill

0xSero

72,465 görüntüleme • 5 ay önce

Gemma4 12B with Unsloth's Quant on DGX Spark Quants: - UD_Q4_K_XL - UD_Q5_K_XL - UD_Q6_K_XL - UD_Q8_K_XL Summary: - Q4: 25.21 tok/s, TTFT 168ms - Q5: 21.7 tok/s, TTFT 182ms - Q6: 17.68 tok/s, TTFT 193.95ms - Q8: 15.22 tok/s, TTFT 221ms
1:03

Gemma4 12B with Unsloth's Quant on DGX Spark Quants: - UD_Q4_K_XL - UD_Q5_K_XL - UD_Q6_K_XL - UD_Q8_K_XL Summary: - Q4: 25.21 tok/s, TTFT 168ms - Q5: 21.7 tok/s, TTFT 182ms - Q6: 17.68 tok/s, TTFT 193.95ms - Q8: 15.22 tok/s, TTFT 221ms

stevibe

18,639 görüntüleme • 3 ay önce

Almost ready, I guess I will finally work on the onboarding experience. I have been using vllm-studio for work all day, this is so good. I legit haven't noticed a single issue with the model, the outputs or even the app's usability. You will have your coding agent at home.
2:15

Almost ready, I guess I will finally work on the onboarding experience. I have been using vllm-studio for work all day, this is so good. I legit haven't noticed a single issue with the model, the outputs or even the app's usability. You will have your coding agent at home.

0xSero

14,320 görüntüleme • 4 ay önce

This is the FASTER MODEL I've EVER Ran LOCALLY 🚀 laguna on Laguna S 2.1 · 3090 Quad INT4 • 1 parallel coding agents • Total: 3,405 tokens in 14s • Aggregate: 256.0 tok/s peak · 249.9 tok/s sustained • Per-stream: 256.0 high / 256.0 low / 256.0 avg tok/s • TTFT 0.12s avg
1:58

This is the FASTER MODEL I've EVER Ran LOCALLY 🚀 laguna on Laguna S 2.1 · 3090 Quad INT4 • 1 parallel coding agents • Total: 3,405 tokens in 14s • Aggregate: 256.0 tok/s peak · 249.9 tok/s sustained • Per-stream: 256.0 high / 256.0 low / 256.0 avg tok/s • TTFT 0.12s avg

Tech2Wild

16,404 görüntüleme • 1 ay önce