Loading video...

Video Failed to Load

Go Home

Demis Hassabis just explained why the real AI bottleneck has nothing to do with training runs. Most people picture the AI arms race as who can build the biggest model. GPT-4 or Gemini Ultra style training runs, a few hundred million in compute, fired once or twice a year....

aakashgupta's profile picture

Aakash Gupta

237,851 subscribers

31,285 views • 2 months ago •via X (Twitter)

News & PoliticsEducationScience & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Jonathan Ross just revealed why AI companies aren’t growing faster. Not demand. Not competition. Physics. Ross: “The demand for compute is insatiable.” There isn’t enough compute in the world. Not a temporary shortage. A fundamental gap between what the market wants and what the infrastructure can deliver. Ross: “Right now, one of the biggest complaints of Anthropic is the rate limits. People can’t get enough tokens.” Rate limits aren’t product decisions. They’re rationing. Companies forced to regulate access because infrastructure cannot meet demand. Slower services. Token caps. The only things standing between these companies and a revenue surge they can’t access. Every token cap is a revenue cap. Every slowdown is a sale that didn’t happen. Ross: “If Anthropic was given twice the inference compute, within one month their revenue would almost double.” Read that again. Double the compute. Double the revenue. Within thirty days. That’s not a growth projection. That’s a measurement of how deep the backlog already is. The demand exists right now. It’s sitting in a queue. The only thing between these companies and that revenue is physical hardware they don’t have. This breaks every assumption about how tech companies scale. Usually you scale by finding customers. AI companies have infinite customers. They scale by finding hardware. The constraint isn’t market fit. It isn’t distribution. It isn’t competition. It’s processing power. This is why Jensen Huang is the most important person in the world right now. NVIDIA doesn’t just make chips. It makes the thing every government, every AI lab, and every company racing for this future needs more of and can’t get enough of. The compute bottleneck isn’t a tech industry problem. It’s a civilizational one. The winner of this era isn’t determined by who builds the smartest model. Every major lab has a frontier model. The winner is whoever secures the most compute fastest while everyone else rations what’s left. The race isn’t for intelligence. It’s for infrastructure. And right now there isn’t enough to go around.
0:53

Jonathan Ross just revealed why AI companies aren’t growing faster. Not demand. Not competition. Physics. Ross: “The demand for compute is insatiable.” There isn’t enough compute in the world. Not a temporary shortage. A fundamental gap between what the market wants and what the infrastructure can deliver. Ross: “Right now, one of the biggest complaints of Anthropic is the rate limits. People can’t get enough tokens.” Rate limits aren’t product decisions. They’re rationing. Companies forced to regulate access because infrastructure cannot meet demand. Slower services. Token caps. The only things standing between these companies and a revenue surge they can’t access. Every token cap is a revenue cap. Every slowdown is a sale that didn’t happen. Ross: “If Anthropic was given twice the inference compute, within one month their revenue would almost double.” Read that again. Double the compute. Double the revenue. Within thirty days. That’s not a growth projection. That’s a measurement of how deep the backlog already is. The demand exists right now. It’s sitting in a queue. The only thing between these companies and that revenue is physical hardware they don’t have. This breaks every assumption about how tech companies scale. Usually you scale by finding customers. AI companies have infinite customers. They scale by finding hardware. The constraint isn’t market fit. It isn’t distribution. It isn’t competition. It’s processing power. This is why Jensen Huang is the most important person in the world right now. NVIDIA doesn’t just make chips. It makes the thing every government, every AI lab, and every company racing for this future needs more of and can’t get enough of. The compute bottleneck isn’t a tech industry problem. It’s a civilizational one. The winner of this era isn’t determined by who builds the smartest model. Every major lab has a frontier model. The winner is whoever secures the most compute fastest while everyone else rations what’s left. The race isn’t for intelligence. It’s for infrastructure. And right now there isn’t enough to go around.

Dustin

28,395 views • 4 months ago

Elon Musk announced a chip factory 10 times the size of Tesla's Gigafactory. The goal is to produce enough AI compute to equal twice the entire electricity consumption of the United States. He called it the Terafab. Here is the number that stopped me cold. The entire global AI chip industry right now is on track to hit around 100 gigawatts per year of compute. Every Nvidia GPU, every Google TPU, every chip from every company on earth combined. 100 gigawatts. Musk wants one factory to produce a terawatt per year. A terawatt is 1,000 gigawatts. Ten times the output of the entire global industry. From a single building. To put the scale in physical terms, the Terafab would need to be around 100 million square feet. You would need Starship point to point transport just to get from one end to the other. But the reason for the scale is not ambition for its own sake. To launch meaningful AI compute into space, you need a billion chips per year running at a kilowatt each. That is not a number the current industry can produce. The Terafab is the only way to get there. The timeline he put out: a gigawatt of space AI compute annualized by end of next year. Then 10x per year from there. 10 gigawatts by year two. 100 gigawatts by year three. A terawatt beyond that. Most people think orbital data centers are a decade away. Musk is building the factory to make them possible by next year.
3:26

Elon Musk announced a chip factory 10 times the size of Tesla's Gigafactory. The goal is to produce enough AI compute to equal twice the entire electricity consumption of the United States. He called it the Terafab. Here is the number that stopped me cold. The entire global AI chip industry right now is on track to hit around 100 gigawatts per year of compute. Every Nvidia GPU, every Google TPU, every chip from every company on earth combined. 100 gigawatts. Musk wants one factory to produce a terawatt per year. A terawatt is 1,000 gigawatts. Ten times the output of the entire global industry. From a single building. To put the scale in physical terms, the Terafab would need to be around 100 million square feet. You would need Starship point to point transport just to get from one end to the other. But the reason for the scale is not ambition for its own sake. To launch meaningful AI compute into space, you need a billion chips per year running at a kilowatt each. That is not a number the current industry can produce. The Terafab is the only way to get there. The timeline he put out: a gigawatt of space AI compute annualized by end of next year. Then 10x per year from there. 10 gigawatts by year two. 100 gigawatts by year three. A terawatt beyond that. Most people think orbital data centers are a decade away. Musk is building the factory to make them possible by next year.

Ihtesham Ali

314,910 views • 11 days ago

Lisa Su just dropped a massive reality check on the state of the AI hardware scramble, and the sheer scale of what is coming is wild. If you asked her six months ago, demand was just "strong." But now? The entire landscape has shifted into absolute overdrive. She specifically noted that in the last 60 to 90 days, there has been a massive acceleration, especially when looking at the forecasts and demand patterns for 2026. The most insane takeaway is her blunt assessment that "there's not enough compute out there for everything that wants to be done." It is a complete bottleneck of global ambition versus available silicon. What is really flying under the radar here is that this isn't exclusively a GPU story anymore. The desire to ramp compute quickly for heavy AI enterprise workloads is actively pulling massive CPU content right along with it. The demand is practically insatiable across the entire board. The heavyweights are scrambling, the trenches are fighting for allocation, and the hardware supercycle is officially accelerating to a whole new level.
1:19

Lisa Su just dropped a massive reality check on the state of the AI hardware scramble, and the sheer scale of what is coming is wild. If you asked her six months ago, demand was just "strong." But now? The entire landscape has shifted into absolute overdrive. She specifically noted that in the last 60 to 90 days, there has been a massive acceleration, especially when looking at the forecasts and demand patterns for 2026. The most insane takeaway is her blunt assessment that "there's not enough compute out there for everything that wants to be done." It is a complete bottleneck of global ambition versus available silicon. What is really flying under the radar here is that this isn't exclusively a GPU story anymore. The desire to ramp compute quickly for heavy AI enterprise workloads is actively pulling massive CPU content right along with it. The demand is practically insatiable across the entire board. The heavyweights are scrambling, the trenches are fighting for allocation, and the hardware supercycle is officially accelerating to a whole new level.

Ian Miles Cheong

100,225 views • 2 months ago

ELON: AI CLUSTERS WILL BECOME A NORMAL THING EVERY COUNTRY HAS Compute is becoming a new currency. While building a "frontier model" today is a high-stakes gauntlet that requires a level of technical skill only a few companies possess, he predicts that AI compute clusters are about to become as essential to a nation as a power grid or a military. By 2026, "AI Sovereignty" will be the only thing keeping nations from becoming digital colonies of Silicon Valley or Beijing: “Probably all countries will have their own AI clusters over time. It is currently very difficult to actually build an AI cluster and have it run... because if you are training a frontier model, then you need a massive amount of compute and a level of technical skill that only a few companies possess. But over time, I think every country will have AI compute clusters. It is just going to be a normal thing that every country has.” Source: vitrupo Elon Musk
1:03

ELON: AI CLUSTERS WILL BECOME A NORMAL THING EVERY COUNTRY HAS Compute is becoming a new currency. While building a "frontier model" today is a high-stakes gauntlet that requires a level of technical skill only a few companies possess, he predicts that AI compute clusters are about to become as essential to a nation as a power grid or a military. By 2026, "AI Sovereignty" will be the only thing keeping nations from becoming digital colonies of Silicon Valley or Beijing: “Probably all countries will have their own AI clusters over time. It is currently very difficult to actually build an AI cluster and have it run... because if you are training a frontier model, then you need a massive amount of compute and a level of technical skill that only a few companies possess. But over time, I think every country will have AI compute clusters. It is just going to be a normal thing that every country has.” Source: vitrupo Elon Musk

Mario Nawfal

20,498 views • 5 months ago

The future of AI compute doesn't run on hype. It runs on real utility, real burns, and a network that gets stronger every time it's used. The Incentive Dynamic Engine (IDE) is now live!
0:31

The future of AI compute doesn't run on hype. It runs on real utility, real burns, and a network that gets stronger every time it's used. The Incentive Dynamic Engine (IDE) is now live!

io.net

9,777,113 views • 14 days ago

$NBIS is becoming one of the biggest winners in the new AI economy because it can deliver compute efficiently at scale. The number that mattered most to me was Nebius seeing four or more customers competing for every GPU it brings online (huge for AI cloud market). Management also said they raised prices again and are still selling out across both old and new GPU generations which is one of the cleanest pricing power signals you can get. This is a $100B company in the making.
0:44

$NBIS is becoming one of the biggest winners in the new AI economy because it can deliver compute efficiently at scale. The number that mattered most to me was Nebius seeing four or more customers competing for every GPU it brings online (huge for AI cloud market). Management also said they raised prices again and are still selling out across both old and new GPU generations which is one of the cleanest pricing power signals you can get. This is a $100B company in the making.

Shay Boloor

495,064 views • 1 month ago

Elon Musk: In the future, every country will have its own AI clusters. “It's currently very difficult to actually build an AI cluster and have it run. It's not like just pulling a computer out of a box. They are currently very difficult to run. And are you going to be training a frontier model? Because if you're training a frontier model, then you need a massive amount of compute and a level of technical skill that only a few companies possess. But, over time, I think every country will have AI compute clusters. It's just going to be a normal thing.” Future Investment Initiative, October 29, 2024
0:34

Elon Musk: In the future, every country will have its own AI clusters. “It's currently very difficult to actually build an AI cluster and have it run. It's not like just pulling a computer out of a box. They are currently very difficult to run. And are you going to be training a frontier model? Because if you're training a frontier model, then you need a massive amount of compute and a level of technical skill that only a few companies possess. But, over time, I think every country will have AI compute clusters. It's just going to be a normal thing.” Future Investment Initiative, October 29, 2024

ELON CLIPS

95,313 views • 7 months ago

Andrew Ng just revealed why the AI companies throwing the most compute at the problem are going to lose. The winner of the intelligence race won’t use the most compute. They’ll waste the least. Ng: “Most of your high-dimensional data lies on a lower-dimensional subspace. It’s just a fact of life.” Here’s what that means in practice. You have a 10,000-dimensional dataset. Every dimension dragged through every calculation. Every training cycle hauling dead weight the model will never use. Ng: “You’re carrying around these 10,000-dimensional examples throughout your whole training process.” That bloat isn’t just inefficient. It’s a tax on every computation you run. Memory bandwidth. Network bandwidth. Computational speed. All of it eaten by dimensions that contribute nothing to intelligence. They contribute noise. The insight that separates the architects from the arms race: that 10,000-dimensional dataset is almost entirely captured by a much smaller subspace. The signal lives in a fraction of the space you’re paying to process. Compress it. 10,000 dimensions down to 1,000. Ng: “You can run your learning algorithm on a much lower-dimensional set of data and it may be much more efficient.” Same hardware. Same budget. A fraction of the friction. Brute force is the strategy of whoever has the deepest pockets. Compression is the strategy of whoever actually understands the problem. The companies that master this don’t just build faster models. They build models that find more truth in less data than anything scaling blindly ever will. Intelligence was never about processing everything. It’s about knowing what to cut.
1:14

Andrew Ng just revealed why the AI companies throwing the most compute at the problem are going to lose. The winner of the intelligence race won’t use the most compute. They’ll waste the least. Ng: “Most of your high-dimensional data lies on a lower-dimensional subspace. It’s just a fact of life.” Here’s what that means in practice. You have a 10,000-dimensional dataset. Every dimension dragged through every calculation. Every training cycle hauling dead weight the model will never use. Ng: “You’re carrying around these 10,000-dimensional examples throughout your whole training process.” That bloat isn’t just inefficient. It’s a tax on every computation you run. Memory bandwidth. Network bandwidth. Computational speed. All of it eaten by dimensions that contribute nothing to intelligence. They contribute noise. The insight that separates the architects from the arms race: that 10,000-dimensional dataset is almost entirely captured by a much smaller subspace. The signal lives in a fraction of the space you’re paying to process. Compress it. 10,000 dimensions down to 1,000. Ng: “You can run your learning algorithm on a much lower-dimensional set of data and it may be much more efficient.” Same hardware. Same budget. A fraction of the friction. Brute force is the strategy of whoever has the deepest pockets. Compression is the strategy of whoever actually understands the problem. The companies that master this don’t just build faster models. They build models that find more truth in less data than anything scaling blindly ever will. Intelligence was never about processing everything. It’s about knowing what to cut.

Dustin

214,927 views • 3 months ago

This guy runs an Nvidia Spark from his desk and pulls in $8,000 a month The device is the size of a small speaker and sits right on his desk. Inside is a full Nvidia GPU running AI workloads 24/7. He rents out the compute power to companies that need local AI processing. They pay him a flat monthly fee to use it remotely. $8,000 lands in his account every single month. He just plugged it in and that was it. The $4,000 box paid for itself in two weeks. Now it prints $96,000 a year from a device smaller than a coffee machine. Companies are desperate for local compute and regular people are quietly supplying it. Save this before everyone figures out what he already knows.
0:42

This guy runs an Nvidia Spark from his desk and pulls in $8,000 a month The device is the size of a small speaker and sits right on his desk. Inside is a full Nvidia GPU running AI workloads 24/7. He rents out the compute power to companies that need local AI processing. They pay him a flat monthly fee to use it remotely. $8,000 lands in his account every single month. He just plugged it in and that was it. The $4,000 box paid for itself in two weeks. Now it prints $96,000 a year from a device smaller than a coffee machine. Companies are desperate for local compute and regular people are quietly supplying it. Save this before everyone figures out what he already knows.

winkle.

57,483 views • 25 days ago

Demis Hassabis confirmed every frontier AI lab is working on recursive self-improvement and in the same sentence said the safety risk of removing humans from the loop entirely keeps him up at night. That combination should stop you. The CEO of Google DeepMind just confirmed that the thing most people treat as a theoretical future risk is already the active focus of every serious lab on earth right now. He explained why it works in coding and math. The feedback loop is fast. You can verify whether an answer is correct almost instantly. You can generate synthetic training data from it. The loop closes quickly and cleanly. Then he said where it breaks down. In biology, chemistry and physics. Any domain where verifying a hypothesis requires a physical experiment in the real world. The loop does not close in seconds. It closes in weeks or months. Geoffrey Hinton said in his Nobel lecture that recursive self-improvement is the development he fears most and that once started it may not be possible to stop. Hassabis is not pushing back on that. He is describing the guardrails labs are building around a process they are already running. Every lab has to think carefully about the safety of a process where no human is in the loop. He said that as a constraint they are navigating right now. The question they are sitting with is how much of it to let run without a human watching. (Watch the full interview on YouTube at Two Minute Papers channel)
2:00

Demis Hassabis confirmed every frontier AI lab is working on recursive self-improvement and in the same sentence said the safety risk of removing humans from the loop entirely keeps him up at night. That combination should stop you. The CEO of Google DeepMind just confirmed that the thing most people treat as a theoretical future risk is already the active focus of every serious lab on earth right now. He explained why it works in coding and math. The feedback loop is fast. You can verify whether an answer is correct almost instantly. You can generate synthetic training data from it. The loop closes quickly and cleanly. Then he said where it breaks down. In biology, chemistry and physics. Any domain where verifying a hypothesis requires a physical experiment in the real world. The loop does not close in seconds. It closes in weeks or months. Geoffrey Hinton said in his Nobel lecture that recursive self-improvement is the development he fears most and that once started it may not be possible to stop. Hassabis is not pushing back on that. He is describing the guardrails labs are building around a process they are already running. Every lab has to think carefully about the safety of a process where no human is in the loop. He said that as a constraint they are navigating right now. The question they are sitting with is how much of it to let run without a human watching. (Watch the full interview on YouTube at Two Minute Papers channel)

Ihtesham Ali

66,950 views • 2 days ago

HRT, one of the largest quantitative trading firms in the world, just revealed the scale of their AI operation. "two of the biggest ingredients you need to build deep learning models are compute and data. HRT has both in incredible orders of magnitude." "we have vast quantities of data, on the order of petabytes, just in terms of raw tick data we've written down since the beginning of electronic trading." "we have a very high compute-per-researcher ratio." then the line that puts it in perspective: "there probably aren't too many places on Earth you can do cutting-edge deep learning research at the level we do it, just because of how few places have access to as much compute." "it's basically the hyperscalers of the world and then we're in the next tranche." a trading firm putting itself in the same compute tier as Google, Microsoft, and Meta. not for chatbots, for trading.
2:08

HRT, one of the largest quantitative trading firms in the world, just revealed the scale of their AI operation. "two of the biggest ingredients you need to build deep learning models are compute and data. HRT has both in incredible orders of magnitude." "we have vast quantities of data, on the order of petabytes, just in terms of raw tick data we've written down since the beginning of electronic trading." "we have a very high compute-per-researcher ratio." then the line that puts it in perspective: "there probably aren't too many places on Earth you can do cutting-edge deep learning research at the level we do it, just because of how few places have access to as much compute." "it's basically the hyperscalers of the world and then we're in the next tranche." a trading firm putting itself in the same compute tier as Google, Microsoft, and Meta. not for chatbots, for trading.

Goshawk Trades

14,548 views • 23 days ago

If intelligence is the log of compute… it starts with a lot of compute! And that’s why we’re scaling our GPU fleet faster than anyone else. Just last year, we added over 2 gigawatts of new capacity – roughly the output of 2 nuclear power plants. And today we’re going further, announcing the world's most powerful AI datacenter, located in southeastern Wisconsin. Fairwater is a seamless cluster of hundreds of thousands of NVIDIA GB200s, connected by enough fiber to circle the Earth 4.5 times. It will deliver 10x the performance of the world’s fastest supercomputer today, enabling AI training and inference workloads at a level never before seen. For AI training workloads, you need compute at exponential scale. That’s why we designed the datacenter, GPU fleet, and network together as one integrated system. This ensures a single job can run from day 1 at exponential scale across thousands of GPUs. Fairwater uses a liquid-cooled closed-loop system for cooling GPUs that requires zero water for operations after construction. And we’re matching all of the energy that is consumed with renewable sources. And of course, it is just one of several similar sites we’re lighting up across our 70+ regions. We have multiple identical Fairwater datacenters under construction in other locations across the US, in addition to our AI infrastructure already deployed in over 100 datacenters around the world, powering model training, test-time compute, RL tuning, and real-time inference at global scale. Too often during times like this, people go with the current and only later wonder, how did we get here? With Fairwater, we're charting a new path: doing the hard engineering work, bringing compute, network, and storage into one highly scaled cluster, and designing closed-loop energy systems to meet real-world computing needs. And partnering with local communities to ensure it's thoughtfully done in a way that is sustainable, creates new jobs, and expands opportunity. We are thrilled to see this take hold in Wisconsin, and we are just getting started.
0:21

If intelligence is the log of compute… it starts with a lot of compute! And that’s why we’re scaling our GPU fleet faster than anyone else. Just last year, we added over 2 gigawatts of new capacity – roughly the output of 2 nuclear power plants. And today we’re going further, announcing the world's most powerful AI datacenter, located in southeastern Wisconsin. Fairwater is a seamless cluster of hundreds of thousands of NVIDIA GB200s, connected by enough fiber to circle the Earth 4.5 times. It will deliver 10x the performance of the world’s fastest supercomputer today, enabling AI training and inference workloads at a level never before seen. For AI training workloads, you need compute at exponential scale. That’s why we designed the datacenter, GPU fleet, and network together as one integrated system. This ensures a single job can run from day 1 at exponential scale across thousands of GPUs. Fairwater uses a liquid-cooled closed-loop system for cooling GPUs that requires zero water for operations after construction. And we’re matching all of the energy that is consumed with renewable sources. And of course, it is just one of several similar sites we’re lighting up across our 70+ regions. We have multiple identical Fairwater datacenters under construction in other locations across the US, in addition to our AI infrastructure already deployed in over 100 datacenters around the world, powering model training, test-time compute, RL tuning, and real-time inference at global scale. Too often during times like this, people go with the current and only later wonder, how did we get here? With Fairwater, we're charting a new path: doing the hard engineering work, bringing compute, network, and storage into one highly scaled cluster, and designing closed-loop energy systems to meet real-world computing needs. And partnering with local communities to ensure it's thoughtfully done in a way that is sustainable, creates new jobs, and expands opportunity. We are thrilled to see this take hold in Wisconsin, and we are just getting started.

Satya Nadella

2,019,532 views • 9 months ago

The CEO of Anthropic just said we are 1 to 3 years away from AGI. On camera. With 90% confidence. Almost nobody watched it. INSTEAD OF ARGUING ABOUT AI TAKING JOBS. Spend 2 hours watching the CEO of Anthropic explain exactly when and how it happens. The AI compute arms race just got a reality check nobody expected. Dario Amodei, CEO of Anthropic just did the math out loud: → Revenue growing 10x per year means $100B by end of 2026 and $1T by end of 2027 → At that trajectory you could theoretically buy $5T in compute over 5 years → But if revenue comes in at even $800B instead of $1T, bankruptcy. No exceptions → And if growth slows from 10x to just 5x per year, the entire model collapses by one year This is the most honest thing anyone in AI has said publicly in years. The companies building AI infrastructure are making trillion-dollar bets on growth curves that have never existed before in history. One year off. One multiplier wrong. The whole thing unravels. The AI compute boom is real. The risk underneath it is equally real. The question is not whether AI is the future. It is whether anyone can survive long enough to get there
2:22:19

The CEO of Anthropic just said we are 1 to 3 years away from AGI. On camera. With 90% confidence. Almost nobody watched it. INSTEAD OF ARGUING ABOUT AI TAKING JOBS. Spend 2 hours watching the CEO of Anthropic explain exactly when and how it happens. The AI compute arms race just got a reality check nobody expected. Dario Amodei, CEO of Anthropic just did the math out loud: → Revenue growing 10x per year means $100B by end of 2026 and $1T by end of 2027 → At that trajectory you could theoretically buy $5T in compute over 5 years → But if revenue comes in at even $800B instead of $1T, bankruptcy. No exceptions → And if growth slows from 10x to just 5x per year, the entire model collapses by one year This is the most honest thing anyone in AI has said publicly in years. The companies building AI infrastructure are making trillion-dollar bets on growth curves that have never existed before in history. One year off. One multiplier wrong. The whole thing unravels. The AI compute boom is real. The risk underneath it is equally real. The question is not whether AI is the future. It is whether anyone can survive long enough to get there

Dami-Defi

16,080 views • 1 month ago

THEY RAN STATE OF THE ART AI ON A 26 YEAR OLD IMAC WITH ZERO INTERNET not a demo. not a benchmark a real answer in near instant time on hardware older than most people reading this your subscription money is going to a data center that doesn't need it the future already runs locally on dead hardware from 1999 bookmark this before you pay for another month of cloud AI you don't need
0:54

THEY RAN STATE OF THE ART AI ON A 26 YEAR OLD IMAC WITH ZERO INTERNET not a demo. not a benchmark a real answer in near instant time on hardware older than most people reading this your subscription money is going to a data center that doesn't need it the future already runs locally on dead hardware from 1999 bookmark this before you pay for another month of cloud AI you don't need

leopardracer

2,761,995 views • 1 month ago

Data is the foundation of any AI training. All of the GPUs in the world can't train an AI model if they don't have data to train it on. Don't forget: Grass is the data layer of AI.
0:40

Data is the foundation of any AI training. All of the GPUs in the world can't train an AI model if they don't have data to train it on. Don't forget: Grass is the data layer of AI.

Grass

211,923 views • 2 years ago

Sergey Brin said compute is dessert. The companies winning the AI race right now are not the ones with the most chips. They are the ones with the best algorithms. Every headline you read about AI is about data centers, Megawatts, and Nvidia orders. Billions in infrastructure and more. The entire investment thesis of the last three years has been built on compute scaling as the primary moat. Sergey thinks that framing is wrong. He pulled out an example from physics. The N-body problem. Scientists have been running those simulations since the fifties. Over the decades, raw compute improved on Moore's law. But the algorithms to solve the problem improved faster. Not slightly faster. Far faster. The algorithmic gains made the compute gains look small. He says the same thing has happened in AI over the last decade. Compute is not the meal. It is the dessert. You still want it. Nobody is turning down frontier compute. But the companies that figured out the algorithms first are the ones actually ahead. The market is pricing AI winners by who has the most chips. Sergey Brin just said that is the wrong scorecard. The ones who win this are not building the biggest data center. They are solving the harder math problem.
1:38

Sergey Brin said compute is dessert. The companies winning the AI race right now are not the ones with the most chips. They are the ones with the best algorithms. Every headline you read about AI is about data centers, Megawatts, and Nvidia orders. Billions in infrastructure and more. The entire investment thesis of the last three years has been built on compute scaling as the primary moat. Sergey thinks that framing is wrong. He pulled out an example from physics. The N-body problem. Scientists have been running those simulations since the fifties. Over the decades, raw compute improved on Moore's law. But the algorithms to solve the problem improved faster. Not slightly faster. Far faster. The algorithmic gains made the compute gains look small. He says the same thing has happened in AI over the last decade. Compute is not the meal. It is the dessert. You still want it. Nobody is turning down frontier compute. But the companies that figured out the algorithms first are the ones actually ahead. The market is pricing AI winners by who has the most chips. Sergey Brin just said that is the wrong scorecard. The ones who win this are not building the biggest data center. They are solving the harder math problem.

Ihtesham Ali

128,769 views • 8 days ago

🚨OpenAI CFO speaks on AI boom and data centers “I think we are sooo in the early innings. I think a lot of prognosticators wanna call it “we’re on the sugar rush”… WE ARE NOT! more like the railroads or the build out of electricity. The internet turns out in hindsight a relatively capex light build out. I think we are just getting started! Now.. we have to do a lot to make data centers more efficient.. we need to think about new ways to power them. BUT in terms of AI, it is voracious right now for GPUs and COMPUTE. the biggest thing we face is WE ARE CONSTANTLY UNDER COMPUTE That’s why we launched Stargate. That’s why we are doing the BIGGER builds with ..Microsoft*, with ORACLE, CoreWeave and so on.. And we are just getting started!”
1:21

🚨OpenAI CFO speaks on AI boom and data centers “I think we are sooo in the early innings. I think a lot of prognosticators wanna call it “we’re on the sugar rush”… WE ARE NOT! more like the railroads or the build out of electricity. The internet turns out in hindsight a relatively capex light build out. I think we are just getting started! Now.. we have to do a lot to make data centers more efficient.. we need to think about new ways to power them. BUT in terms of AI, it is voracious right now for GPUs and COMPUTE. the biggest thing we face is WE ARE CONSTANTLY UNDER COMPUTE That’s why we launched Stargate. That’s why we are doing the BIGGER builds with ..Microsoft*, with ORACLE, CoreWeave and so on.. And we are just getting started!”

NIK

102,475 views • 10 months ago

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast
1:29

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast

AI Evolution

15,565 views • 25 days ago

Dylan Patel, founder of SemiAnalysis: "The upper bound on how much compute can be produced by 2030 is around 200 gigawatts a year." The entire world has about 20 gigawatts of AI deployed right now. The ceiling is 10x what exists today, and it still isn't enough to feed what Sam, Elon, Dario and Demis are racing to build. Everyone argues about which model wins. The real limit is a number measured in gigawatts, and it's already maxed out years in advance.
7:13

Dylan Patel, founder of SemiAnalysis: "The upper bound on how much compute can be produced by 2030 is around 200 gigawatts a year." The entire world has about 20 gigawatts of AI deployed right now. The ceiling is 10x what exists today, and it still isn't enough to feed what Sam, Elon, Dario and Demis are racing to build. Everyone argues about which model wins. The real limit is a number measured in gigawatts, and it's already maxed out years in advance.

zostaff

163,567 views • 13 days ago

Flapping Airplanes co-founder Asher Spector explains why data efficiency is the greatest bottleneck to AI adoption: "To the extent that AI has been hard to integrate into the economy, I really think it's because models are much less data-efficient than humans. If you want it to learn a new task, or put it in a new vertical, it takes thousands of times more effort than it does to just tell a human what to do." "If you can make a model a million times more data-efficient, it's a million times easier to put into the economy. There's a ton of cool stuff that you can do in really data-constrained regimes. For example, whether it's robotics, or scientific discovery, or even something like trading, these problems have very limited data, and existing AI systems aren't quite as good at them as they are at other things. I think that learning to learn with less data is just tremendously valuable in all of these domains."
1:44

Flapping Airplanes co-founder Asher Spector explains why data efficiency is the greatest bottleneck to AI adoption: "To the extent that AI has been hard to integrate into the economy, I really think it's because models are much less data-efficient than humans. If you want it to learn a new task, or put it in a new vertical, it takes thousands of times more effort than it does to just tell a human what to do." "If you can make a model a million times more data-efficient, it's a million times easier to put into the economy. There's a ton of cool stuff that you can do in really data-constrained regimes. For example, whether it's robotics, or scientific discovery, or even something like trading, these problems have very limited data, and existing AI systems aren't quite as good at them as they are at other things. I think that learning to learn with less data is just tremendously valuable in all of these domains."

TBPN

89,394 views • 4 months ago