正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Sergey Brin on Scaling Laws vs. algorithms: “Compute is the dessert. Algorithms are the main course. Algorithmic progress has outpaced compute and data scaling over the last decade.” This echoes Ilya’s view: scaling what we have will keep leading to improvements, but some key algorithmic breakthrough is still missing... show more

Yuchen Jin

94,380 subscribers

90,818 次观看 • 5 个月前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Microsoft CEO Satya Nadella says the AI scaling laws multiply intelligence in both compute and algorithmic dimensions: for every 10x increase in compute, capability increases 100x

Microsoft CEO Satya Nadella says the AI scaling laws multiply intelligence in both compute and algorithmic dimensions: for every 10x increase in compute, capability increases 100x

Tsarathustra

80,676 次观看 • 1 年前

"within 2-5 years, we’ll unlock the embodied scaling law for robotics" Nvidia Senior Research Scientist, Jim Fan: this breakthrough will clarify how compute investments translate into robot capabilities, just as scaling laws did for LLMs

"within 2-5 years, we’ll unlock the embodied scaling law for robotics" Nvidia Senior Research Scientist, Jim Fan: this breakthrough will clarify how compute investments translate into robot capabilities, just as scaling laws did for LLMs

Haider.

26,715 次观看 • 1 年前

$AGI timelines are very bimodal. It's either by 2030 or bust. AI progress over the last decade has been driven by scaling training compute of frontier systems (3.55x a year, 160x over 4 years). This simply cannot continue beyond this decade, whether you look at chips, power, even fraction of raw GDP used on training. After 2030, AI progress has to mostly come from algorithmic progress. But even there the low hanging fruit will be plucked (at least under the deep learning paradigm). So the yearly probability of AGI craters. And we're plausibly looking at 2040+ timelines. I discuss this dynamic with Sholto Douglas and Trenton Bricken:$

AGI timelines are very bimodal. It's either by 2030 or bust. AI progress over the last decade has been driven by scaling training compute of frontier systems (3.55x a year, 160x over 4 years). This simply cannot continue beyond this decade, whether you look at chips, power, even fraction of raw GDP used on training. After 2030, AI progress has to mostly come from algorithmic progress. But even there the low hanging fruit will be plucked (at least under the deep learning paradigm). So the yearly probability of AGI craters. And we're plausibly looking at 2040+ timelines. I discuss this dynamic with Sholto Douglas and Trenton Bricken:

Dwarkesh Patel

220,361 次观看 • 1 年前

For years, the scaling law turned AI into an industrial arms race. Academia couldn’t compete on compute. But if Ilya is right and scaling is plateauing, the pendulum may swing back to pure research. What if the next breakthrough comes not from a hyperscaler but from 2 PhD students scribbling on a university blackboard?

For years, the scaling law turned AI into an industrial arms race. Academia couldn’t compete on compute. But if Ilya is right and scaling is plateauing, the pendulum may swing back to pure research. What if the next breakthrough comes not from a hyperscaler but from 2 PhD students scribbling on a university blackboard?

Luis Batalha

23,458 次观看 • 6 个月前

Greg Brockman says OpenAI started with a belief that AGI would come from ideas, not compute But by 2017, scaling became the clearest path: 2x compute made agents 2x better 10 GW is a drop in the bucket compared to the compute needed for the AGI vision

Greg Brockman says OpenAI started with a belief that AGI would come from ideas, not compute But by 2017, scaling became the clearest path: 2x compute made agents 2x better 10 GW is a drop in the bucket compared to the compute needed for the AGI vision

Haider.

89,012 次观看 • 8 个月前

Microsoft CEO Satya Nadella says that rather than seeing AI Scaling Laws hit a wall, if anything we are seeing the emergence of a new Scaling Law for test-time compute

Microsoft CEO Satya Nadella says that rather than seeing AI Scaling Laws hit a wall, if anything we are seeing the emergence of a new Scaling Law for test-time compute

Tsarathustra

111,422 次观看 • 1 年前

OpenAI initially focused on ideas for AGI, but empirical results in 2017 showed scaling compute was key. Seeing 2x improvements in DOTA 2 with 2x scale, they pushed limits, advising chip startups on future models. Internalizing compute became vital to realizing their vision.

OpenAI initially focused on ideas for AGI, but empirical results in 2017 showed scaling compute was key. Seeing 2x improvements in DOTA 2 with 2x scale, they pushed limits, advising chip startups on future models. Internalizing compute became vital to realizing their vision.

Chubby♨️

12,597 次观看 • 8 个月前

Geoffrey Hinton says the real breakthrough in AI wasn’t just new algorithms, but massive scaling of data and computation. Ilya Sutskever predicted early that simply making models bigger would make them better, and he turned out to be right. While new ideas like transformers helped, the billion-fold increase in compute power and larger datasets drove most of the progress.

Geoffrey Hinton says the real breakthrough in AI wasn’t just new algorithms, but massive scaling of data and computation. Ilya Sutskever predicted early that simply making models bigger would make them better, and he turned out to be right. While new ideas like transformers helped, the billion-fold increase in compute power and larger datasets drove most of the progress.

Wes Roth

14,768 次观看 • 10 个月前

There is no limit: Dario Amodei says that even without better algorithms, scaling will continue to lead to amazing improvements in AI models

There is no limit: Dario Amodei says that even without better algorithms, scaling will continue to lead to amazing improvements in AI models

Tsarathustra

157,362 次观看 • 2 年前

What could scaling unlock for biology? Introducing ProGen3- our next AI foundation models for protein generation. We develop compute-optimal scaling laws up to 46B parameters on 1.5T tokens with real evidence in the wet lab. +we solve a new set of challenges for drug discovery

What could scaling unlock for biology? Introducing ProGen3- our next AI foundation models for protein generation. We develop compute-optimal scaling laws up to 46B parameters on 1.5T tokens with real evidence in the wet lab. +we solve a new set of challenges for drug discovery

Ali Madani

80,512 次观看 • 1 年前

1/ Introducing: The Nexus 1.0 machine. We are scaling ZK to 1 trillion Hz of compute capacity.

1/ Introducing: The Nexus 1.0 machine. We are scaling ZK to 1 trillion Hz of compute capacity.

Nexus

68,526 次观看 • 2 年前

Dwarkesh says the Age of Scaling isn’t over. Companies will keep pouring exponentially more compute into training because the value of intelligence is so high, it’s worth even $1 trillion if it gets us to AGI. Right now, compute usage for frontier models grows ~4x per year, but by 2028–2030, physical and economic limits (chips, energy, GDP) will take progress slower. A 160x increase in four years can’t go on.

Dwarkesh says the Age of Scaling isn’t over. Companies will keep pouring exponentially more compute into training because the value of intelligence is so high, it’s worth even $1 trillion if it gets us to AGI. Right now, compute usage for frontier models grows ~4x per year, but by 2028–2030, physical and economic limits (chips, energy, GDP) will take progress slower. A 160x increase in four years can’t go on.

Wes Roth

18,531 次观看 • 11 个月前

Ex-OpenAI researcher: "Scaling laws don't fail. What fails is the data feeding them.”

Ex-OpenAI researcher: "Scaling laws don't fail. What fails is the data feeding them.”

Lior Alexander

37,338 次观看 • 4 个月前

"it would surprise me if Human-level AI is still decades away" Nick Bostrom explains: if reaching AGI depends on scaling compute with larger data centers, progress may slow due to cost and infrastructure limits. however, the leap could come much faster if a new algorithm drastically improves efficiency.

"it would surprise me if Human-level AI is still decades away" Nick Bostrom explains: if reaching AGI depends on scaling compute with larger data centers, progress may slow due to cost and infrastructure limits. however, the leap could come much faster if a new algorithm drastically improves efficiency.

Haider.

20,236 次观看 • 1 年前

OpenAI's Noam Brown says letting AI models think for 20 seconds got the same boost in performance as scaling up the model by 100,000x and training it for 100,000 times longer, and scaling model size and compute will end up being a footnote compared to scaling model thinking

OpenAI's Noam Brown says letting AI models think for 20 seconds got the same boost in performance as scaling up the model by 100,000x and training it for 100,000 times longer, and scaling model size and compute will end up being a footnote compared to scaling model thinking

Tsarathustra

273,449 次观看 • 1 年前

We’re no longer just scaling computing power. We’re using compute to scale intelligence itself. That’s what makes this moment historically significant. For sixty years, progress in computing followed Moore’s Law—transistor density doubling roughly every two years. But AI is advancing on a far steeper curve. Today, frontier model capabilities are improving on a cadence closer to every six months—an order of magnitude faster than classical hardware scaling. The underlying principle is both simple and radical: when you increase data, compute, and model complexity, intelligence emerges. Scaling laws show that larger models—given sufficient compute and high-quality data—become predictably more capable. In just over a decade, we’ve gone from neural nets that could identify cats to systems that can draft legal briefs, write production-grade code, generate scientific hypotheses, and outperform top human competitors in mathematics, strategy, and reasoning tasks. This is no longer “software” in the traditional sense. It is a new form of intelligence—synthetic, scalable, rapidly compounding, and increasingly able to take meaningful action in the real world. The geopolitical, economic, and societal implications of this shift are only beginning to unfold—and they will redefine global power in the decades ahead.

We’re no longer just scaling computing power. We’re using compute to scale intelligence itself. That’s what makes this moment historically significant. For sixty years, progress in computing followed Moore’s Law—transistor density doubling roughly every two years. But AI is advancing on a far steeper curve. Today, frontier model capabilities are improving on a cadence closer to every six months—an order of magnitude faster than classical hardware scaling. The underlying principle is both simple and radical: when you increase data, compute, and model complexity, intelligence emerges. Scaling laws show that larger models—given sufficient compute and high-quality data—become predictably more capable. In just over a decade, we’ve gone from neural nets that could identify cats to systems that can draft legal briefs, write production-grade code, generate scientific hypotheses, and outperform top human competitors in mathematics, strategy, and reasoning tasks. This is no longer “software” in the traditional sense. It is a new form of intelligence—synthetic, scalable, rapidly compounding, and increasingly able to take meaningful action in the real world. The geopolitical, economic, and societal implications of this shift are only beginning to unfold—and they will redefine global power in the decades ahead.

Nina Schick

17,722 次观看 • 5 个月前

Jensen Huang: OpenAI & Anthropic are struggling to keep up with demand. The *exponential* growth in AI compute, adoption, and applications is immense - never seen in humanity. Supporting their scaling is critical.

Jensen Huang: OpenAI & Anthropic are struggling to keep up with demand. The exponential growth in AI compute, adoption, and applications is immense - never seen in humanity. Supporting their scaling is critical.

Chubby♨️

44,006 次观看 • 6 个月前