Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Magic CEO Eric Steinberger says there is only one remaining problem to be solved before reaching AGI and that is spending more inference time compute per token

tsarnick's profile picture

Tsarathustra

85,968 subscribers

127,556 Aufrufe • vor 1 Jahr •via X (Twitter)

Nachrichten & PolitikWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Tsarathustra
Tsarathustravor 1 Jahr

Source:

Profilbild von Elon Musk
Elon Muskvor 1 Jahr

That is an important part of the solution

Profilbild von Tsarathustra
Tsarathustravor 1 Jahr

Colossus is purely for training, right? Will there be an equivalent inference cluster?

Profilbild von Knut Jägersberg
Knut Jägersbergvor 1 Jahr

now, while their active reasoning approach makes me curious, I'm not sure if simply smart inference schemes are gonna make the critical difference. system 2 is not just about spending effort and revising thoughts. I sense their approach might lack 'feeling' among others.

Profilbild von Tsarathustra
Tsarathustravor 1 Jahr

feelings are just a system for prioritizing needs

Profilbild von Aidan McLaughlin
Aidan McLaughlinvor 1 Jahr

L

Profilbild von Yossi Dahan
Yossi Dahanvor 1 Jahr

While I respect the Magic CEO, claiming there’s only one remaining hurdle before AGI seems a bit overconfident. The leaders of major labs tend to be more cautious, providing extensive disclaimers with their AGI predictions. It might be wise to add a bit of humility and nuance to such claims.

Profilbild von Shawn
Shawnvor 1 Jahr

More time at inference doesn't change the weights, which means the model does not "understand." Sure it will help with known data but not novel data

Profilbild von JJ
JJvor 1 Jahr

🤣😂😅🤣🤣🤣😂😂 What???? This guy..

Profilbild von Micah
Micahvor 1 Jahr

I don't trust any AI company CEOs with a monetary incentive to spread the hype. Analysts who have no financial involvement are much more reliable IMO.

Ähnliche Videos

IN NEWS: Baseten raises a $150M series D round. Tuhin Srivastava (Founder & CEO, Baseten) on the future of inference: “I think the token price goes down and inference should get cheaper over time. And that really just means there is going to be more inference.” “Every time we lower prices or optimize models to make it cheaper, four months later customers are spending more anyway.” “Inference prices will go down, but if the world is run by AI in 10 years, there is going to be a lot of inference. It better be cheap.”
0:52

IN NEWS: Baseten raises a $150M series D round. Tuhin Srivastava (Founder & CEO, Baseten) on the future of inference: “I think the token price goes down and inference should get cheaper over time. And that really just means there is going to be more inference.” “Every time we lower prices or optimize models to make it cheaper, four months later customers are spending more anyway.” “Inference prices will go down, but if the world is run by AI in 10 years, there is going to be a lot of inference. It better be cheap.”

TBPN

17,057 Aufrufe • vor 10 Monaten

A muslim migrant woman is complaining that there are too many Finnish people in Finland. She further says that this problem can be solved in future with more muslim migrants coming in.
0:21

A muslim migrant woman is complaining that there are too many Finnish people in Finland. She further says that this problem can be solved in future with more muslim migrants coming in.

Incognito

93,812 Aufrufe • vor 1 Jahr

"People do not realize that there is only one bridge remaining." To solve disease + accelerate, we need to make disease solvable using compute. Data is the limiting reagent. This is why Precigenetics is essential for the future of biological superintelligence.
2:32

"People do not realize that there is only one bridge remaining." To solve disease + accelerate, we need to make disease solvable using compute. Data is the limiting reagent. This is why Precigenetics is essential for the future of biological superintelligence.

Parmita Mishra

89,821 Aufrufe • vor 4 Monaten

Mamdani’s “solution” to housing is nonsensical and would make everything far worse. But many conservatives refuse to acknowledge that there even is a problem — or that it’s NOT one that can be solved by simply not buying lattes.
1:58

Mamdani’s “solution” to housing is nonsensical and would make everything far worse. But many conservatives refuse to acknowledge that there even is a problem — or that it’s NOT one that can be solved by simply not buying lattes.

Kat Timpf

240,538 Aufrufe • vor 6 Monaten

We did it. We finally did it. Infinite memory. Microsoft AI CEO Mustafa Suleyman: “We have prototypes that have near-infinite memory. And so it just doesn’t forget, which is truly transformative. (...) 2025 is going to be truly transformative” Eric Schmidt (Ex-Google CEO) wasn't lying when he said infinite context window will be solved this year. Thats it, AGI is so close.
1:27

We did it. We finally did it. Infinite memory. Microsoft AI CEO Mustafa Suleyman: “We have prototypes that have near-infinite memory. And so it just doesn’t forget, which is truly transformative. (...) 2025 is going to be truly transformative” Eric Schmidt (Ex-Google CEO) wasn't lying when he said infinite context window will be solved this year. Thats it, AGI is so close.

Chubby♨️

261,858 Aufrufe • vor 1 Jahr

“NVIDIA’s cost per token is the lowest in the world.” — Jensen Huang, Founder & CEO of NVIDIA Token generation cost is a direct result of architecture excellence and extreme co-design, not just compute cost. Lowest cost per token and highest performance per watt are definitive measures of AI economics and the key to unlocking maximum profitability and AI revenue. ➡️
0:49

“NVIDIA’s cost per token is the lowest in the world.” — Jensen Huang, Founder & CEO of NVIDIA Token generation cost is a direct result of architecture excellence and extreme co-design, not just compute cost. Lowest cost per token and highest performance per watt are definitive measures of AI economics and the key to unlocking maximum profitability and AI revenue. ➡️

NVIDIA

88,885 Aufrufe • vor 3 Monaten

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!
0:39

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Milk Road AI

128,079 Aufrufe • vor 15 Tagen

.Arvind Krishna, IBM CEO, says there is only one solution to data security we should be looking at: quantum-proof encryption. Watch #wef23 here:
0:20

.Arvind Krishna, IBM CEO, says there is only one solution to data security we should be looking at: quantum-proof encryption. Watch #wef23 here:

World Economic Forum

29,523 Aufrufe • vor 3 Jahren

"Inference, if you look at it as a market, will be much, much bigger than cloud computing was pre-ChatGPT." Lightspeed’s Bucky Moore says inference is an underrated investment category in AI, and expects the market to break up into large, specialized platforms for each modality: "The GPU supply crunch that we're seeing right now is largely, as Dylan Patel has said on the show before, due to the fact that not only these consumer products, but also B2B products like Claude Code and Codex are just really taking off and creating insane demand for inference." "We're talking hundreds of billions in spend every year. And if that's true, I think there will be very, very large inference platforms built in each modality." "So there will be an inference platform for real-time video models, there will be an inference platform for open-source and custom language models, there will be an inference platform built specifically for long-running agents." "So I think we're just going to see that industry, which today looks like one industry, break up into many because of how big it is and how much room for specialization there is."
1:02

"Inference, if you look at it as a market, will be much, much bigger than cloud computing was pre-ChatGPT." Lightspeed’s Bucky Moore says inference is an underrated investment category in AI, and expects the market to break up into large, specialized platforms for each modality: "The GPU supply crunch that we're seeing right now is largely, as Dylan Patel has said on the show before, due to the fact that not only these consumer products, but also B2B products like Claude Code and Codex are just really taking off and creating insane demand for inference." "We're talking hundreds of billions in spend every year. And if that's true, I think there will be very, very large inference platforms built in each modality." "So there will be an inference platform for real-time video models, there will be an inference platform for open-source and custom language models, there will be an inference platform built specifically for long-running agents." "So I think we're just going to see that industry, which today looks like one industry, break up into many because of how big it is and how much room for specialization there is."

TBPN

33,278 Aufrufe • vor 3 Monaten

There is only one Paulson Stadium! Only one Statesboro, GA! It is time to GATA JUST ONE MORE TIME! Night game! 6pm! Pack Paulson!!!
0:50

There is only one Paulson Stadium! Only one Statesboro, GA! It is time to GATA JUST ONE MORE TIME! Night game! 6pm! Pack Paulson!!!

Clay Helton

19,831 Aufrufe • vor 1 Jahr

Every AI product running in production depends on an inference system someone had to engineer, optimize and more. Scheduling, batching, routing, cost per token. This is a craft. The Inference Frontier Program spotlights the builders behind that work. 💡 Watch the video and nominate a team:
1:22

Every AI product running in production depends on an inference system someone had to engineer, optimize and more. Scheduling, batching, routing, cost per token. This is a craft. The Inference Frontier Program spotlights the builders behind that work. 💡 Watch the video and nominate a team:

Nebius

54,224 Aufrufe • vor 4 Monaten

They have no respect for anything or anyone because they know there will be no consequences. This is a huge problem that needs to be solved.
0:38

They have no respect for anything or anyone because they know there will be no consequences. This is a huge problem that needs to be solved.

RadioGenoa

495,313 Aufrufe • vor 2 Jahren

"It just doesn't compute. And I think when things like this don't compute, it means that there is going to be a very fast reconciliation at some point, which is the Omega Candle and a fast run to $1 million [bitcoin]." - Samson Mow, CEO of JAN3
0:59

"It just doesn't compute. And I think when things like this don't compute, it means that there is going to be a very fast reconciliation at some point, which is the Omega Candle and a fast run to $1 million [bitcoin]." - Samson Mow, CEO of JAN3

THE Bitcoin Podcast

37,704 Aufrufe • vor 11 Monaten

🚨xAI: WE’RE BUILDING THE FASTEST, SMARTEST AGI OUT THERE "As a company, xAI, we are going to be the fastest-moving AGI company out there. Next, we will continue developing a model that is not just intelligent, not just smart, not just capable of sustained thinking and high compute usage, but a model that is both fast and smart. That will be our core focus." Source xAI
0:43

🚨xAI: WE’RE BUILDING THE FASTEST, SMARTEST AGI OUT THERE "As a company, xAI, we are going to be the fastest-moving AGI company out there. Next, we will continue developing a model that is not just intelligent, not just smart, not just capable of sustained thinking and high compute usage, but a model that is both fast and smart. That will be our core focus." Source xAI

Mario Nawfal

46,851 Aufrufe • vor 1 Jahr

Don't be tricked by this math problem! There are 2 common answers to this but only one is correct. Which is it and why?
2:17

Don't be tricked by this math problem! There are 2 common answers to this but only one is correct. Which is it and why?

Howie Hua

10,405 Aufrufe • vor 1 Jahr

When there is only one more sleep before you're off to Australia and New Zealand. #letsgobuntingmental #WorldSeries #pdc
0:46

When there is only one more sleep before you're off to Australia and New Zealand. #letsgobuntingmental #WorldSeries #pdc

Stephen Bunting

22,268 Aufrufe • vor 11 Monaten