Загрузка видео...

Не удалось загрузить видео

На главную

Hot take: Tokens per second is a completely meaningless metric

AdamHoltererer's profile picture

Adam Holter

3,438 subscribers

302,013 просмотров • 7 дней назад •via X (Twitter)

Здоровье и велнесНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

"50 tokens per second is too slow" The slow 👇
0:26

"50 tokens per second is too slow" The slow 👇

Javier • priv/acc

11,219 просмотров • 1 месяц назад

This is from my conversation with a Former $NVDA Director. If you want more TOKENS per SECOND because you are limited by POWER, you buy the next $NVDA generation. - Limited to 100 MW: $NVDA H100 = 300M tokens/second - Limited to 100 MW: $NVDA GB200 NVL72 = 12B tokens/second
2:22

This is from my conversation with a Former $NVDA Director. If you want more TOKENS per SECOND because you are limited by POWER, you buy the next $NVDA generation. - Limited to 100 MW: $NVDA H100 = 300M tokens/second - Limited to 100 MW: $NVDA GB200 NVL72 = 12B tokens/second

Rihard Jarc

57,622 просмотров • 1 год назад

Nvidia's Chief Scientist Bill Dally says there's a path to serving relatively large models at 10,000 to 20,000 tokens per user per second. For context, Opus 4.6 is ~43 and Grok 4.2 Beta is ~251 tokens/user/s 🤯
1:30

Nvidia's Chief Scientist Bill Dally says there's a path to serving relatively large models at 10,000 to 20,000 tokens per user per second. For context, Opus 4.6 is ~43 and Grok 4.2 Beta is ~251 tokens/user/s 🤯

Marcelo P. Lima

84,684 просмотров • 6 месяцев назад

My mind is blown. GroqInc is serving LLaMA 3 at over 800 tokens per second! 800. Tokens. Per. Second. This unlocks so many incredible use-cases. It's one thing to see my demo — it's another thing entirely to experience it for yourself. Do yourself a favor and try it asap.
0:21

My mind is blown. GroqInc is serving LLaMA 3 at over 800 tokens per second! 800. Tokens. Per. Second. This unlocks so many incredible use-cases. It's one thing to see my demo — it's another thing entirely to experience it for yourself. Do yourself a favor and try it asap.

Matt Shumer

589,626 просмотров • 2 лет назад

“AI COULD SOON THINK 100× TO 1,000× FASTER THAN HUMANS.” Emad Mostaque says AI systems are about to become dramatically faster as specialized AI hardware improves. He says humans currently operate at roughly 20–30 tokens per second, while current AI models are around 40 tokens per second. He points to Taalas' technology, which he says demonstrated an 8-billion-parameter model generating around 14,000 tokens per second. Mostaque says current AI systems are effectively “thinking at the speed of a human”, but that will soon change. He predicts AI systems could reach 1,000 tokens per second, followed by 10,000 tokens per second by the end of next year. His broader point is that AI could eventually “think a thousand times faster” than humans.
1:58

“AI COULD SOON THINK 100× TO 1,000× FASTER THAN HUMANS.” Emad Mostaque says AI systems are about to become dramatically faster as specialized AI hardware improves. He says humans currently operate at roughly 20–30 tokens per second, while current AI models are around 40 tokens per second. He points to Taalas' technology, which he says demonstrated an 8-billion-parameter model generating around 14,000 tokens per second. Mostaque says current AI systems are effectively “thinking at the speed of a human”, but that will soon change. He predicts AI systems could reach 1,000 tokens per second, followed by 10,000 tokens per second by the end of next year. His broader point is that AI could eventually “think a thousand times faster” than humans.

SciTech Era

12,824 просмотров • 1 месяц назад

15,000 tokens per second feels absolutely insane. Is this the future? This is if you haven't heard of it from Taalas Inc..
0:23

15,000 tokens per second feels absolutely insane. Is this the future? This is if you haven't heard of it from Taalas Inc..

David Gomes

262,584 просмотров • 5 месяцев назад

Everyone’s obsessed with cheaper tokens. Meanwhile, enterprise AI spend keeps going up. Chamath says costs are doubling every 45 days for ~5% productivity gains. Glean’s founder says the metric that actually matters is cost per successful task, not cost per token.
1:09

Everyone’s obsessed with cheaper tokens. Meanwhile, enterprise AI spend keeps going up. Chamath says costs are doubling every 45 days for ~5% productivity gains. Glean’s founder says the metric that actually matters is cost per successful task, not cost per token.

Not Jerome Powell

45,215 просмотров • 2 месяцев назад

Qwen3.5-35B-A3B running locally on an M4 chip at 49.5 tokens per second. A 35B model. On a laptop. In real time. LOCAL AI IS GETTING SCARY FAST.
1:10

Qwen3.5-35B-A3B running locally on an M4 chip at 49.5 tokens per second. A 35B model. On a laptop. In real time. LOCAL AI IS GETTING SCARY FAST.

0xMarioNawfal

478,419 просмотров • 7 месяцев назад

Watch Gemma-4-26B running 18 concurrent sessions on a single NVIDIA AI DGX Spark — delivering a cumulative 300 tokens per second. 🔥
0:53

Watch Gemma-4-26B running 18 concurrent sessions on a single NVIDIA AI DGX Spark — delivering a cumulative 300 tokens per second. 🔥

Mia

23,221 просмотров • 3 месяцев назад

“Energy density per unit raw material” is the right metric, and the winner is nuclear fission!
2:55

“Energy density per unit raw material” is the right metric, and the winner is nuclear fission!

Isaiah Taylor - making nuclear reactors

27,632 просмотров • 1 год назад

Gemma 4 running locally on a Nintendo Switch :) 1.5 tokens per second haha, but it runs! Google Gemma Google AI Developers Google DeepMind
2:57

Gemma 4 running locally on a Nintendo Switch :) 1.5 tokens per second haha, but it runs! Google Gemma Google AI Developers Google DeepMind

Maddie D. Reese

188,407 просмотров • 6 месяцев назад

.Ahmad stopped by MTS with a NVIDIA DGX Station (Dell Technologies Pro Max with GB300). 🔥 Thousands of tokens per second all running locally.
0:58

.Ahmad stopped by MTS with a NVIDIA DGX Station (Dell Technologies Pro Max with GB300). 🔥 Thousands of tokens per second all running locally.

NVIDIA RTX Spark

12,130 просмотров • 25 дней назад

This is Ultrafast. Our premium speed tier, Ultrafast offers up to 8x faster token generation (300 tokens per second) in Codex and up to 6x in the API.
0:45

This is Ultrafast. Our premium speed tier, Ultrafast offers up to 8x faster token generation (300 tokens per second) in Codex and up to 6x in the API.

OpenAI

1,358,798 просмотров • 13 дней назад

Hayek explains why social justice is a meaningless term: “Justice is an attribute of individual action. I can be just or unjust towards my fellow men.” “But the conception of a social justice—to expect a just result from an impersonal process which nobody can control—is meaningless and completely impossible.”
0:42

Hayek explains why social justice is a meaningless term: “Justice is an attribute of individual action. I can be just or unjust towards my fellow men.” “But the conception of a social justice—to expect a just result from an impersonal process which nobody can control—is meaningless and completely impossible.”

F. A. Hayek Quotes

33,349 просмотров • 5 месяцев назад