Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Sergey Brin on Scaling Laws vs. algorithms: “Compute is the dessert. Algorithms are the main course. Algorithmic progress has outpaced compute and data scaling over the last decade.” This echoes Ilya’s view: scaling what we have will keep leading to improvements, but some key algorithmic breakthrough is still missing... show more

Yuchen Jin

94,380 subscribers

90,818 views • 7 months ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Sergey Brin said compute is dessert. The companies winning the AI race right now are not the ones with the most chips. They are the ones with the best algorithms. Every headline you read about AI is about data centers, Megawatts, and Nvidia orders. Billions in infrastructure and more. The entire investment thesis of the last three years has been built on compute scaling as the primary moat. Sergey thinks that framing is wrong. He pulled out an example from physics. The N-body problem. Scientists have been running those simulations since the fifties. Over the decades, raw compute improved on Moore's law. But the algorithms to solve the problem improved faster. Not slightly faster. Far faster. The algorithmic gains made the compute gains look small. He says the same thing has happened in AI over the last decade. Compute is not the meal. It is the dessert. You still want it. Nobody is turning down frontier compute. But the companies that figured out the algorithms first are the ones actually ahead. The market is pricing AI winners by who has the most chips. Sergey Brin just said that is the wrong scorecard. The ones who win this are not building the biggest data center. They are solving the harder math problem.

Sergey Brin said compute is dessert. The companies winning the AI race right now are not the ones with the most chips. They are the ones with the best algorithms. Every headline you read about AI is about data centers, Megawatts, and Nvidia orders. Billions in infrastructure and more. The entire investment thesis of the last three years has been built on compute scaling as the primary moat. Sergey thinks that framing is wrong. He pulled out an example from physics. The N-body problem. Scientists have been running those simulations since the fifties. Over the decades, raw compute improved on Moore's law. But the algorithms to solve the problem improved faster. Not slightly faster. Far faster. The algorithmic gains made the compute gains look small. He says the same thing has happened in AI over the last decade. Compute is not the meal. It is the dessert. You still want it. Nobody is turning down frontier compute. But the companies that figured out the algorithms first are the ones actually ahead. The market is pricing AI winners by who has the most chips. Sergey Brin just said that is the wrong scorecard. The ones who win this are not building the biggest data center. They are solving the harder math problem.

Ihtesham Ali

130,048 views • 1 month ago

$AGI timelines are very bimodal. It's either by 2030 or bust. AI progress over the last decade has been driven by scaling training compute of frontier systems (3.55x a year, 160x over 4 years). This simply cannot continue beyond this decade, whether you look at chips, power, even fraction of raw GDP used on training. After 2030, AI progress has to mostly come from algorithmic progress. But even there the low hanging fruit will be plucked (at least under the deep learning paradigm). So the yearly probability of AGI craters. And we're plausibly looking at 2040+ timelines. I discuss this dynamic with Sholto Douglas and Trenton Bricken:$

AGI timelines are very bimodal. It's either by 2030 or bust. AI progress over the last decade has been driven by scaling training compute of frontier systems (3.55x a year, 160x over 4 years). This simply cannot continue beyond this decade, whether you look at chips, power, even fraction of raw GDP used on training. After 2030, AI progress has to mostly come from algorithmic progress. But even there the low hanging fruit will be plucked (at least under the deep learning paradigm). So the yearly probability of AGI craters. And we're plausibly looking at 2040+ timelines. I discuss this dynamic with Sholto Douglas and Trenton Bricken:

Dwarkesh Patel

220,473 views • 1 year ago

For years, the scaling law turned AI into an industrial arms race. Academia couldn’t compete on compute. But if Ilya is right and scaling is plateauing, the pendulum may swing back to pure research. What if the next breakthrough comes not from a hyperscaler but from 2 PhD students scribbling on a university blackboard?

For years, the scaling law turned AI into an industrial arms race. Academia couldn’t compete on compute. But if Ilya is right and scaling is plateauing, the pendulum may swing back to pure research. What if the next breakthrough comes not from a hyperscaler but from 2 PhD students scribbling on a university blackboard?

Luis Batalha

23,458 views • 8 months ago

Geoffrey Hinton says the real breakthrough in AI wasn’t just new algorithms, but massive scaling of data and computation. Ilya Sutskever predicted early that simply making models bigger would make them better, and he turned out to be right. While new ideas like transformers helped, the billion-fold increase in compute power and larger datasets drove most of the progress.

Geoffrey Hinton says the real breakthrough in AI wasn’t just new algorithms, but massive scaling of data and computation. Ilya Sutskever predicted early that simply making models bigger would make them better, and he turned out to be right. While new ideas like transformers helped, the billion-fold increase in compute power and larger datasets drove most of the progress.

Wes Roth

14,768 views • 1 year ago

Dwarkesh says the Age of Scaling isn’t over. Companies will keep pouring exponentially more compute into training because the value of intelligence is so high, it’s worth even $1 trillion if it gets us to AGI. Right now, compute usage for frontier models grows ~4x per year, but by 2028–2030, physical and economic limits (chips, energy, GDP) will take progress slower. A 160x increase in four years can’t go on.

Dwarkesh says the Age of Scaling isn’t over. Companies will keep pouring exponentially more compute into training because the value of intelligence is so high, it’s worth even $1 trillion if it gets us to AGI. Right now, compute usage for frontier models grows ~4x per year, but by 2028–2030, physical and economic limits (chips, energy, GDP) will take progress slower. A 160x increase in four years can’t go on.

Wes Roth

18,531 views • 1 year ago

We’re no longer just scaling computing power. We’re using compute to scale intelligence itself. That’s what makes this moment historically significant. For sixty years, progress in computing followed Moore’s Law—transistor density doubling roughly every two years. But AI is advancing on a far steeper curve. Today, frontier model capabilities are improving on a cadence closer to every six months—an order of magnitude faster than classical hardware scaling. The underlying principle is both simple and radical: when you increase data, compute, and model complexity, intelligence emerges. Scaling laws show that larger models—given sufficient compute and high-quality data—become predictably more capable. In just over a decade, we’ve gone from neural nets that could identify cats to systems that can draft legal briefs, write production-grade code, generate scientific hypotheses, and outperform top human competitors in mathematics, strategy, and reasoning tasks. This is no longer “software” in the traditional sense. It is a new form of intelligence—synthetic, scalable, rapidly compounding, and increasingly able to take meaningful action in the real world. The geopolitical, economic, and societal implications of this shift are only beginning to unfold—and they will redefine global power in the decades ahead.

We’re no longer just scaling computing power. We’re using compute to scale intelligence itself. That’s what makes this moment historically significant. For sixty years, progress in computing followed Moore’s Law—transistor density doubling roughly every two years. But AI is advancing on a far steeper curve. Today, frontier model capabilities are improving on a cadence closer to every six months—an order of magnitude faster than classical hardware scaling. The underlying principle is both simple and radical: when you increase data, compute, and model complexity, intelligence emerges. Scaling laws show that larger models—given sufficient compute and high-quality data—become predictably more capable. In just over a decade, we’ve gone from neural nets that could identify cats to systems that can draft legal briefs, write production-grade code, generate scientific hypotheses, and outperform top human competitors in mathematics, strategy, and reasoning tasks. This is no longer “software” in the traditional sense. It is a new form of intelligence—synthetic, scalable, rapidly compounding, and increasingly able to take meaningful action in the real world. The geopolitical, economic, and societal implications of this shift are only beginning to unfold—and they will redefine global power in the decades ahead.

Nina Schick

17,739 views • 6 months ago

China has a new export: AI compute Data companies are shipping processed tokens overseas via undersea cables. While the US builds closed $200/mo models, China is building AI for the world -- accessible, open, and scaling fast

China has a new export: AI compute Data companies are shipping processed tokens overseas via undersea cables. While the US builds closed $200/mo models, China is building AI for the world -- accessible, open, and scaling fast

COMBATE |🇵🇷

75,020 views • 1 month ago

Ilya Sutskever just dropped a bombshell: The scaling era is DEAD. "Scaling is just one word" — but from 2020–2025 it brainwashed the entire industry. Pour more compute + data → predictable gains. Low-risk gold rush for companies. Now? Data is finite. Returns are flattening hard. The easy money party is over — we're crashing back into the age of research, with giant computers that won't save you if the ideas suck. 2:04 clip — wake-up call?

Ilya Sutskever just dropped a bombshell: The scaling era is DEAD. "Scaling is just one word" — but from 2020–2025 it brainwashed the entire industry. Pour more compute + data → predictable gains. Low-risk gold rush for companies. Now? Data is finite. Returns are flattening hard. The easy money party is over — we're crashing back into the age of research, with giant computers that won't save you if the ideas suck. 2:04 clip — wake-up call?

Camus

21,422 views • 6 months ago

📣 Call for proposals! Our first funded programme is now live — Scaling Compute: AI at 1/1000th the cost. Hear more from PD Suraj Bramhavar on how we aim to radically reduce the cost + energy consumption of compute hardware, and open new avenues to reap the benefits of AI 👇

📣 Call for proposals! Our first funded programme is now live — Scaling Compute: AI at 1/1000th the cost. Hear more from PD Suraj Bramhavar on how we aim to radically reduce the cost + energy consumption of compute hardware, and open new avenues to reap the benefits of AI 👇

ARIA

61,132 views • 2 years ago

“People are already building things that weren’t possible before.” With Mainnet and TGE live, Acurast is focused on what comes next: from scaling real-world compute to new infrastructure use cases, including Agentic AI. Watch the founders’ update 👇

“People are already building things that weren’t possible before.” With Mainnet and TGE live, Acurast is focused on what comes next: from scaling real-world compute to new infrastructure use cases, including Agentic AI. Watch the founders’ update 👇

Acurast

26,828 views • 6 months ago

ELON MUSK: “To reach a terawatt of compute per year, we need about 10 million tons to orbit annually. This is feasible—no new physics required—and I’m confident SpaceX can achieve it. We’re also scaling to a terawatt of solar for power. The remaining piece is a terawatt of compute, which is why we’re building Terafab.”

ELON MUSK: “To reach a terawatt of compute per year, we need about 10 million tons to orbit annually. This is feasible—no new physics required—and I’m confident SpaceX can achieve it. We’re also scaling to a terawatt of solar for power. The remaining piece is a terawatt of compute, which is why we’re building Terafab.”

Tesla Owners Silicon Valley

12,017 views • 1 month ago

Our last AI episode with Jonathan Ross @ Groq exploded and there was just one piece of feedback everyone had: “Get Andrew Feldman on next.” So we did. And it breaks more conventional rules on AI than ever before 📈 Scaling laws are total BS 📉 AI algorithms are so inefficient ⚔️ NVIDIA is not suited to a world of inference 😮 CUDA is not defensible for NVIDIA More high fives in this than the Wiz boardroom and my 8 key takeaways 👇

Our last AI episode with Jonathan Ross @ Groq exploded and there was just one piece of feedback everyone had: “Get Andrew Feldman on next.” So we did. And it breaks more conventional rules on AI than ever before 📈 Scaling laws are total BS 📉 AI algorithms are so inefficient ⚔️ NVIDIA is not suited to a world of inference 😮 CUDA is not defensible for NVIDIA More high fives in this than the Wiz boardroom and my 8 key takeaways 👇

Harry Stebbings

13,158 views • 1 year ago

Roman Yampolskiy says AGI is no longer a question of when, but how much it costs Scaling laws are predictable: the cost has dropped from $1 trillion to $100 billion, and will soon reach $1 billion AI already writes 100% of the code for next-gen models, so recursive self-improvement has already begun

Roman Yampolskiy says AGI is no longer a question of when, but how much it costs Scaling laws are predictable: the cost has dropped from $1 trillion to $100 billion, and will soon reach $1 billion AI already writes 100% of the code for next-gen models, so recursive self-improvement has already begun

Haider.

18,844 views • 4 months ago

Sundar Pichai said six months ago that Google had already been experiencing real energy bottlenecks in scaling AI. Yesterday, $GOOG announced the acquisition of Intersect Power, which operates some of the largest solar farms in the U.S. This clearly shows how significant an energy bottleneck is for scaling AI. Generation and transmission capacity are the main bottlenecks for data centers connected to the grid, while permitting and availability of strategically positioned land with access to energy sources and water are the main obstacles for on-site generation. Companies addressing these bottlenecks will do very well in the next decade. $TLN $BE $PWR $ET $TPL $LB $CEG $MTZ $DY $GEV

Sundar Pichai said six months ago that Google had already been experiencing real energy bottlenecks in scaling AI. Yesterday, $GOOG announced the acquisition of Intersect Power, which operates some of the largest solar farms in the U.S. This clearly shows how significant an energy bottleneck is for scaling AI. Generation and transmission capacity are the main bottlenecks for data centers connected to the grid, while permitting and availability of strategically positioned land with access to energy sources and water are the main obstacles for on-site generation. Companies addressing these bottlenecks will do very well in the next decade. $TLN $BE $PWR $ET $TPL $LB $CEG $MTZ $DY $GEV

Oguz Erkan

179,657 views • 7 months ago

“Private smart contracts are the last big missing piece of what our industry is supposed to supply to the financial system." Sergey Nazarov explains how Chainlink Confidential Compute and CRE enable institutions to create private smart contracts on any chain ↓

“Private smart contracts are the last big missing piece of what our industry is supposed to supply to the financial system." Sergey Nazarov explains how Chainlink Confidential Compute and CRE enable institutions to create private smart contracts on any chain ↓

Chainlink

59,145 views • 8 months ago