Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Sergey Brin on Scaling Laws vs. algorithms: “Compute is the dessert. Algorithms are the main course. Algorithmic progress has outpaced compute and data scaling over the last decade.” This echoes Ilya’s view: scaling what we have will keep leading to improvements, but some key algorithmic breakthrough is still missing... show more

Yuchen Jin

94,380 subscribers

90,818 Aufrufe • vor 6 Monaten •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Sergey Brin said compute is dessert. The companies winning the AI race right now are not the ones with the most chips. They are the ones with the best algorithms. Every headline you read about AI is about data centers, Megawatts, and Nvidia orders. Billions in infrastructure and more. The entire investment thesis of the last three years has been built on compute scaling as the primary moat. Sergey thinks that framing is wrong. He pulled out an example from physics. The N-body problem. Scientists have been running those simulations since the fifties. Over the decades, raw compute improved on Moore's law. But the algorithms to solve the problem improved faster. Not slightly faster. Far faster. The algorithmic gains made the compute gains look small. He says the same thing has happened in AI over the last decade. Compute is not the meal. It is the dessert. You still want it. Nobody is turning down frontier compute. But the companies that figured out the algorithms first are the ones actually ahead. The market is pricing AI winners by who has the most chips. Sergey Brin just said that is the wrong scorecard. The ones who win this are not building the biggest data center. They are solving the harder math problem.

Sergey Brin said compute is dessert. The companies winning the AI race right now are not the ones with the most chips. They are the ones with the best algorithms. Every headline you read about AI is about data centers, Megawatts, and Nvidia orders. Billions in infrastructure and more. The entire investment thesis of the last three years has been built on compute scaling as the primary moat. Sergey thinks that framing is wrong. He pulled out an example from physics. The N-body problem. Scientists have been running those simulations since the fifties. Over the decades, raw compute improved on Moore's law. But the algorithms to solve the problem improved faster. Not slightly faster. Far faster. The algorithmic gains made the compute gains look small. He says the same thing has happened in AI over the last decade. Compute is not the meal. It is the dessert. You still want it. Nobody is turning down frontier compute. But the companies that figured out the algorithms first are the ones actually ahead. The market is pricing AI winners by who has the most chips. Sergey Brin just said that is the wrong scorecard. The ones who win this are not building the biggest data center. They are solving the harder math problem.

Ihtesham Ali

129,786 Aufrufe • vor 25 Tagen

Microsoft CEO Satya Nadella says the AI scaling laws multiply intelligence in both compute and algorithmic dimensions: for every 10x increase in compute, capability increases 100x

Microsoft CEO Satya Nadella says the AI scaling laws multiply intelligence in both compute and algorithmic dimensions: for every 10x increase in compute, capability increases 100x

Tsarathustra

80,676 Aufrufe • vor 1 Jahr

$AGI timelines are very bimodal. It's either by 2030 or bust. AI progress over the last decade has been driven by scaling training compute of frontier systems (3.55x a year, 160x over 4 years). This simply cannot continue beyond this decade, whether you look at chips, power, even fraction of raw GDP used on training. After 2030, AI progress has to mostly come from algorithmic progress. But even there the low hanging fruit will be plucked (at least under the deep learning paradigm). So the yearly probability of AGI craters. And we're plausibly looking at 2040+ timelines. I discuss this dynamic with Sholto Douglas and Trenton Bricken:$

AGI timelines are very bimodal. It's either by 2030 or bust. AI progress over the last decade has been driven by scaling training compute of frontier systems (3.55x a year, 160x over 4 years). This simply cannot continue beyond this decade, whether you look at chips, power, even fraction of raw GDP used on training. After 2030, AI progress has to mostly come from algorithmic progress. But even there the low hanging fruit will be plucked (at least under the deep learning paradigm). So the yearly probability of AGI craters. And we're plausibly looking at 2040+ timelines. I discuss this dynamic with Sholto Douglas and Trenton Bricken:

Dwarkesh Patel

220,473 Aufrufe • vor 1 Jahr

For years, the scaling law turned AI into an industrial arms race. Academia couldn’t compete on compute. But if Ilya is right and scaling is plateauing, the pendulum may swing back to pure research. What if the next breakthrough comes not from a hyperscaler but from 2 PhD students scribbling on a university blackboard?

For years, the scaling law turned AI into an industrial arms race. Academia couldn’t compete on compute. But if Ilya is right and scaling is plateauing, the pendulum may swing back to pure research. What if the next breakthrough comes not from a hyperscaler but from 2 PhD students scribbling on a university blackboard?

Luis Batalha

23,458 Aufrufe • vor 7 Monaten

Greg Brockman says OpenAI started with a belief that AGI would come from ideas, not compute But by 2017, scaling became the clearest path: 2x compute made agents 2x better 10 GW is a drop in the bucket compared to the compute needed for the AGI vision

Greg Brockman says OpenAI started with a belief that AGI would come from ideas, not compute But by 2017, scaling became the clearest path: 2x compute made agents 2x better 10 GW is a drop in the bucket compared to the compute needed for the AGI vision

Haider.

89,012 Aufrufe • vor 9 Monaten

Microsoft CEO Satya Nadella says that rather than seeing AI Scaling Laws hit a wall, if anything we are seeing the emergence of a new Scaling Law for test-time compute

Microsoft CEO Satya Nadella says that rather than seeing AI Scaling Laws hit a wall, if anything we are seeing the emergence of a new Scaling Law for test-time compute

Tsarathustra

111,422 Aufrufe • vor 1 Jahr

OpenAI initially focused on ideas for AGI, but empirical results in 2017 showed scaling compute was key. Seeing 2x improvements in DOTA 2 with 2x scale, they pushed limits, advising chip startups on future models. Internalizing compute became vital to realizing their vision.

OpenAI initially focused on ideas for AGI, but empirical results in 2017 showed scaling compute was key. Seeing 2x improvements in DOTA 2 with 2x scale, they pushed limits, advising chip startups on future models. Internalizing compute became vital to realizing their vision.

Chubby♨️

12,597 Aufrufe • vor 9 Monaten

Geoffrey Hinton says the real breakthrough in AI wasn’t just new algorithms, but massive scaling of data and computation. Ilya Sutskever predicted early that simply making models bigger would make them better, and he turned out to be right. While new ideas like transformers helped, the billion-fold increase in compute power and larger datasets drove most of the progress.

Geoffrey Hinton says the real breakthrough in AI wasn’t just new algorithms, but massive scaling of data and computation. Ilya Sutskever predicted early that simply making models bigger would make them better, and he turned out to be right. While new ideas like transformers helped, the billion-fold increase in compute power and larger datasets drove most of the progress.

Wes Roth

14,768 Aufrufe • vor 11 Monaten

There is no limit: Dario Amodei says that even without better algorithms, scaling will continue to lead to amazing improvements in AI models

There is no limit: Dario Amodei says that even without better algorithms, scaling will continue to lead to amazing improvements in AI models

Tsarathustra

157,362 Aufrufe • vor 2 Jahren

What could scaling unlock for biology? Introducing ProGen3- our next AI foundation models for protein generation. We develop compute-optimal scaling laws up to 46B parameters on 1.5T tokens with real evidence in the wet lab. +we solve a new set of challenges for drug discovery

What could scaling unlock for biology? Introducing ProGen3- our next AI foundation models for protein generation. We develop compute-optimal scaling laws up to 46B parameters on 1.5T tokens with real evidence in the wet lab. +we solve a new set of challenges for drug discovery

Ali Madani

80,642 Aufrufe • vor 1 Jahr

1/ Introducing: The Nexus 1.0 machine. We are scaling ZK to 1 trillion Hz of compute capacity.

1/ Introducing: The Nexus 1.0 machine. We are scaling ZK to 1 trillion Hz of compute capacity.

Nexus

68,543 Aufrufe • vor 2 Jahren

Dwarkesh says the Age of Scaling isn’t over. Companies will keep pouring exponentially more compute into training because the value of intelligence is so high, it’s worth even $1 trillion if it gets us to AGI. Right now, compute usage for frontier models grows ~4x per year, but by 2028–2030, physical and economic limits (chips, energy, GDP) will take progress slower. A 160x increase in four years can’t go on.

Dwarkesh says the Age of Scaling isn’t over. Companies will keep pouring exponentially more compute into training because the value of intelligence is so high, it’s worth even $1 trillion if it gets us to AGI. Right now, compute usage for frontier models grows ~4x per year, but by 2028–2030, physical and economic limits (chips, energy, GDP) will take progress slower. A 160x increase in four years can’t go on.

Wes Roth

18,531 Aufrufe • vor 1 Jahr

Ex-OpenAI researcher: "Scaling laws don't fail. What fails is the data feeding them.”

Ex-OpenAI researcher: "Scaling laws don't fail. What fails is the data feeding them.”

Lior Alexander

37,349 Aufrufe • vor 5 Monaten

OpenAI's Noam Brown says letting AI models think for 20 seconds got the same boost in performance as scaling up the model by 100,000x and training it for 100,000 times longer, and scaling model size and compute will end up being a footnote compared to scaling model thinking

OpenAI's Noam Brown says letting AI models think for 20 seconds got the same boost in performance as scaling up the model by 100,000x and training it for 100,000 times longer, and scaling model size and compute will end up being a footnote compared to scaling model thinking

Tsarathustra

273,500 Aufrufe • vor 1 Jahr

We’re no longer just scaling computing power. We’re using compute to scale intelligence itself. That’s what makes this moment historically significant. For sixty years, progress in computing followed Moore’s Law—transistor density doubling roughly every two years. But AI is advancing on a far steeper curve. Today, frontier model capabilities are improving on a cadence closer to every six months—an order of magnitude faster than classical hardware scaling. The underlying principle is both simple and radical: when you increase data, compute, and model complexity, intelligence emerges. Scaling laws show that larger models—given sufficient compute and high-quality data—become predictably more capable. In just over a decade, we’ve gone from neural nets that could identify cats to systems that can draft legal briefs, write production-grade code, generate scientific hypotheses, and outperform top human competitors in mathematics, strategy, and reasoning tasks. This is no longer “software” in the traditional sense. It is a new form of intelligence—synthetic, scalable, rapidly compounding, and increasingly able to take meaningful action in the real world. The geopolitical, economic, and societal implications of this shift are only beginning to unfold—and they will redefine global power in the decades ahead.

We’re no longer just scaling computing power. We’re using compute to scale intelligence itself. That’s what makes this moment historically significant. For sixty years, progress in computing followed Moore’s Law—transistor density doubling roughly every two years. But AI is advancing on a far steeper curve. Today, frontier model capabilities are improving on a cadence closer to every six months—an order of magnitude faster than classical hardware scaling. The underlying principle is both simple and radical: when you increase data, compute, and model complexity, intelligence emerges. Scaling laws show that larger models—given sufficient compute and high-quality data—become predictably more capable. In just over a decade, we’ve gone from neural nets that could identify cats to systems that can draft legal briefs, write production-grade code, generate scientific hypotheses, and outperform top human competitors in mathematics, strategy, and reasoning tasks. This is no longer “software” in the traditional sense. It is a new form of intelligence—synthetic, scalable, rapidly compounding, and increasingly able to take meaningful action in the real world. The geopolitical, economic, and societal implications of this shift are only beginning to unfold—and they will redefine global power in the decades ahead.

Nina Schick

17,728 Aufrufe • vor 6 Monaten