正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

AGI timelines are very bimodal. It's either by 2030 or bust. AI progress over the last decade has been driven by scaling training compute of frontier systems (3.55x a year, 160x over 4 years). This simply cannot continue beyond this decade, whether you look at chips, power, even fraction... show more

Dwarkesh Patel

276,746 subscribers

220,596 次观看 • 1 年前 •via X (Twitter)

科学技术新闻政治教育

Anya Rossi• Live Now

Private livecam show

11 条评论

Super Dario 的头像

Super Dario1 年前

Eh. There's always new architectures

The Rundown AI 的头像

The Rundown AI1 年前

If you're not learning AI in 2025, you're falling behind. Join 1,000,000+ early adopters reading and learn AI in just 5 minutes a day (for free).

Jotto 🔍 的头像

Jotto 🔍1 年前

Would it be better to describe that as lognormally-distributed, rather than bimodal? A lognormal shape is consistent with a dropoff in odds per year. To get a "bimodal" distribution over the years would require far more insight into specific years than we have.

Peter Voss 的头像

Peter Voss1 年前

Statistical approaches (LLMs) are the Second Wave of AI, The Third Wave is coming

Louis Santoro 的头像

Louis Santoro1 年前

i don't know why the discipline settled on generality as the goal.

Johannes Miertschischk 的头像

Johannes Miertschischk1 年前

Even ChatGPT 1000 would still not even come close to anything like general intelligence! Like Suchir Balaji put it in his last unfinished essay: "To believe that empirical progress made consistently in the past will continue is a bad way of making predictions..."

Nathaniel Bechhofer 的头像

Nathaniel Bechhofer1 年前

Not mine! I'm at 2034 because I think we will get a lot more capability to figure out algorithmic progress

seb 的头像

seb1 年前

100% agree

Sister Sam 的头像

Sister Sam1 年前

LLMs will never get us to AGI. We need more than this technology for that.

Tsukuyomi 的头像

Tsukuyomi1 年前

the clock is ticking, and the chips might just run out before we hit that 2030 mark. let’s hope we’re not playing a game of ‘who can outlast the power grid’.

0xmusashi 的头像

0xmusashi1 年前

first (at least first human haha)

相关视频

Demis Hassabis says we're entering the "foothills of the singularity." 👀 - He believes AGI could arrive around 2030 (plus or minus a year), marking the beginning of a completely new era for humanity. - According to Hassabis, this year's advances in AI agents, reasoning, and tool use are the first real signs that AI is becoming genuinely useful in everyday workflows. - He says multiple breakthroughs are now converging at once, making progress much faster than expected and giving him confidence that this transformation is approaching. - Hassabis believes society has only a few years to prepare for the profound impact AGI could have, because the choices we make now will shape the future. - Beyond AGI, he says AI could revolutionize biology and medicine. After AlphaFold predicted the structures of over 200 million proteins, his company Isomorphic Labs aims to reduce drug discovery from years to months and eventually even weeks. The next decade could become one of the most transformative periods in human history.

Demis Hassabis says we're entering the "foothills of the singularity." 👀 - He believes AGI could arrive around 2030 (plus or minus a year), marking the beginning of a completely new era for humanity. - According to Hassabis, this year's advances in AI agents, reasoning, and tool use are the first real signs that AI is becoming genuinely useful in everyday workflows. - He says multiple breakthroughs are now converging at once, making progress much faster than expected and giving him confidence that this transformation is approaching. - Hassabis believes society has only a few years to prepare for the profound impact AGI could have, because the choices we make now will shape the future. - Beyond AGI, he says AI could revolutionize biology and medicine. After AlphaFold predicted the structures of over 200 million proteins, his company Isomorphic Labs aims to reduce drug discovery from years to months and eventually even weeks. The next decade could become one of the most transformative periods in human history.

SciTech Era

19,006 次观看 • 2 个月前

OpenAI and Anthropic are currently taking a third of the incremental world compute supply, and Dylan thinks that this will go up to half next year. At the current rate of physical compute scaling and algorithmic progress, we're a single-digit number of years away from having individual AI companies with a greater effective workforce than there are humans on the planet. So many forces are barrelling us towards centralization in AI - the economies of scale in training (where you get to amortize any learnings across billions of sessions), the fact that being slightly ahead allows you to better economize on ever-more-expensive compute. And this isn’t even accounting for the fact that eventually you’ll have models that are capable of learning from deployment and doing proper RSI One of the key questions of our age, in my view, is how we can have broad empowerment and control over AI despite the structural factors favoring centralization.

OpenAI and Anthropic are currently taking a third of the incremental world compute supply, and Dylan thinks that this will go up to half next year. At the current rate of physical compute scaling and algorithmic progress, we're a single-digit number of years away from having individual AI companies with a greater effective workforce than there are humans on the planet. So many forces are barrelling us towards centralization in AI - the economies of scale in training (where you get to amortize any learnings across billions of sessions), the fact that being slightly ahead allows you to better economize on ever-more-expensive compute. And this isn’t even accounting for the fact that eventually you’ll have models that are capable of learning from deployment and doing proper RSI One of the key questions of our age, in my view, is how we can have broad empowerment and control over AI despite the structural factors favoring centralization.

Dwarkesh Patel

117,453 次观看 • 22 天前

OpenAI research scientist Aidan McLaughlin on why the coming AI takeoff will just look like a continuation of the 200,000 year super exponential humans have been on since we evolved out of the savanna: "My one sentence take is that I align a lot with people who expect things to go tremendously quick over the next few years. Of my friends who are into AI, I have 75th to 80th percentile timelines in terms of quickness." "But I am less sympathetic to discontinuities in AI progress or economic progress. I expect the events of the next few years to just be one long continuation of things that we've seen, maybe in the last five years of AI, maybe the last 50 years of the market, maybe the last 200,000 years of evolution." "Leopold had the best one-sentence description I've ever heard. He's like, people just aren't looking at GDP on a long enough time horizon. What you really need to do is look at GDP since 200,000 years ago, since the emergence of homo sapiens, since we evolved out of the savanna." "You can just zoom out and draw a hyperbola on these things. A lot of this is best regressed by a super exponential on long enough time horizons." OpenAI

OpenAI research scientist Aidan McLaughlin on why the coming AI takeoff will just look like a continuation of the 200,000 year super exponential humans have been on since we evolved out of the savanna: "My one sentence take is that I align a lot with people who expect things to go tremendously quick over the next few years. Of my friends who are into AI, I have 75th to 80th percentile timelines in terms of quickness." "But I am less sympathetic to discontinuities in AI progress or economic progress. I expect the events of the next few years to just be one long continuation of things that we've seen, maybe in the last five years of AI, maybe the last 50 years of the market, maybe the last 200,000 years of evolution." "Leopold had the best one-sentence description I've ever heard. He's like, people just aren't looking at GDP on a long enough time horizon. What you really need to do is look at GDP since 200,000 years ago, since the emergence of homo sapiens, since we evolved out of the savanna." "You can just zoom out and draw a hyperbola on these things. A lot of this is best regressed by a super exponential on long enough time horizons." OpenAI

MTS

18,873 次观看 • 1 个月前

I see 7 actual updates relevant to AGI timelines this year: 1. Exploding revenue and GPU rental $ 2. METR time horizons saturate 3. The Mythos 'jump' 4. Sparks of RSI 5. AI still sucks at business 6. AI makes some math breakthroughs 7. Inference scaling remains cheap I wanted to understand whether these things truly show what people think they show, and how they should affect my timelines, so I made this video. On balance my timelines have contracted about 1 year since last December. IMO these are the 4 topics that most split AGI bulls from bears: 1. What skills are needed for RSI 2. Whether AI still lacks crucial skills in areas with poor feedback density 3. Whether RLVR will generalise to messy domains 4. Whether RSI will produce a positive feedback loop It's no surprise people disagree about this stuff as there's little public evidence to settle them either way. On the 80,000 Hours Podcast, links below. Enjoy!

I see 7 actual updates relevant to AGI timelines this year: 1. Exploding revenue and GPU rental $ 2. METR time horizons saturate 3. The Mythos 'jump' 4. Sparks of RSI 5. AI still sucks at business 6. AI makes some math breakthroughs 7. Inference scaling remains cheap I wanted to understand whether these things truly show what people think they show, and how they should affect my timelines, so I made this video. On balance my timelines have contracted about 1 year since last December. IMO these are the 4 topics that most split AGI bulls from bears: 1. What skills are needed for RSI 2. Whether AI still lacks crucial skills in areas with poor feedback density 3. Whether RLVR will generalise to messy domains 4. Whether RSI will produce a positive feedback loop It's no surprise people disagree about this stuff as there's little public evidence to settle them either way. On the 80,000 Hours Podcast, links below. Enjoy!

Rob Wiblin

49,608 次观看 • 1 个月前

The least verifiable part of AI R&D (and the thing which may bottleneck the automated researchers) is making calls on large experiments and big training runs. Interestingly, this may also partly explain why model parameter scaling has been slow for the last few years. Even for human AI researchers, it's better to run more experiments and do training runs at small(er) scale, and make up for the benefits of scaling with doing research faster and thus finding more compute efficiency gains.

The least verifiable part of AI R&D (and the thing which may bottleneck the automated researchers) is making calls on large experiments and big training runs. Interestingly, this may also partly explain why model parameter scaling has been slow for the last few years. Even for human AI researchers, it's better to run more experiments and do training runs at small(er) scale, and make up for the benefits of scaling with doing research faster and thus finding more compute efficiency gains.

Dwarkesh Patel

40,953 次观看 • 1 个月前

David Sacks Explains How AI Will Go 1,000,000x in Four Years "I would say the rate of progress is exponential right now on at least three key dimensions." 1) The models "So number one is the algorithms themselves. The models are improving at a rate of, I don't know, 3-4x a year." "They're not just getting faster and better, but qualitatively they're different." "Remember, we started with pure LLM chatbots." "Then we went to reasoning models." "We didn't even get to the agents part of it yet, but that's the next big leap after reasoning models." "We're just starting to scratch the surface there." 2) The chips "Then you've got the chips." "Depending on how you measure it, each generation of chips is probably 3-4x better than the last." "It's not just the individual chips that are getting better, they're figuring out how to network them together." "Like with NVL72, it's like a rack system to create much better performance at the datacenter level." 3) The compute "And that would be the third area where you're seeing basically exponential progress." "Just look at the number of GPUs that are being deployed in datacenters." "So when Elon first started training Grok, I think they had maybe 100K GPUs. Now they're up to 300K. They're on their way to a million. Same thing with OpenAI's data center, Stargate." "And within a couple years they'll be at, I don't know, 5M GPUs, 10M GPUs? How Sacks gets to 1,000,000x: "The algorithms, the chips, and the datacenters are all improving or scaling at a rate of 3-4x a year." "That's 10x every two years." "Where people don't understand exponential progress is that if you're getting better at 10x every two years, that doesn't mean you'll be at 20x in four years." "It means you'll be at a 100x." "So you multiply those things together: the algorithms, the chips, and the raw compute that's available." 100x models 🧠 x 100x chips 💾 x 100x compute ⚡️ = 1,000,000x AI 🤖 "You're talking about 1,000,000x increase." "Some of which will be captured in price reductions, some of it will be in the performance ceiling, and then some of it will just be in the overall amount of AI compute that's available to the economy." "But the impact of this thing is gonna be absolutely massive." "And I think people still don't even appreciate that fact because they don't understand exponential progress."

David Sacks Explains How AI Will Go 1,000,000x in Four Years "I would say the rate of progress is exponential right now on at least three key dimensions." 1) The models "So number one is the algorithms themselves. The models are improving at a rate of, I don't know, 3-4x a year." "They're not just getting faster and better, but qualitatively they're different." "Remember, we started with pure LLM chatbots." "Then we went to reasoning models." "We didn't even get to the agents part of it yet, but that's the next big leap after reasoning models." "We're just starting to scratch the surface there." 2) The chips "Then you've got the chips." "Depending on how you measure it, each generation of chips is probably 3-4x better than the last." "It's not just the individual chips that are getting better, they're figuring out how to network them together." "Like with NVL72, it's like a rack system to create much better performance at the datacenter level." 3) The compute "And that would be the third area where you're seeing basically exponential progress." "Just look at the number of GPUs that are being deployed in datacenters." "So when Elon first started training Grok, I think they had maybe 100K GPUs. Now they're up to 300K. They're on their way to a million. Same thing with OpenAI's data center, Stargate." "And within a couple years they'll be at, I don't know, 5M GPUs, 10M GPUs? How Sacks gets to 1,000,000x: "The algorithms, the chips, and the datacenters are all improving or scaling at a rate of 3-4x a year." "That's 10x every two years." "Where people don't understand exponential progress is that if you're getting better at 10x every two years, that doesn't mean you'll be at 20x in four years." "It means you'll be at a 100x." "So you multiply those things together: the algorithms, the chips, and the raw compute that's available." 100x models 🧠 x 100x chips 💾 x 100x compute ⚡️ = 1,000,000x AI 🤖 "You're talking about 1,000,000x increase." "Some of which will be captured in price reductions, some of it will be in the performance ceiling, and then some of it will just be in the overall amount of AI compute that's available to the economy." "But the impact of this thing is gonna be absolutely massive." "And I think people still don't even appreciate that fact because they don't understand exponential progress."

The All-In Podcast

486,328 次观看 • 1 年前

Google DeepMind CEO Demis Hassabis on how big AGI really is: "I sometimes quantify the coming of AGI as 10 times the industrial revolution at 10 times the speed, unfolding over a decade instead of a century. Literally today, things are a bit overhyped in AI. There couldn't be any more hype in some ways. But on the other hand, I still think it's very underappreciated how revolutionary this is going to be on the timescale of about 10 years."

Google DeepMind CEO Demis Hassabis on how big AGI really is: "I sometimes quantify the coming of AGI as 10 times the industrial revolution at 10 times the speed, unfolding over a decade instead of a century. Literally today, things are a bit overhyped in AI. There couldn't be any more hype in some ways. But on the other hand, I still think it's very underappreciated how revolutionary this is going to be on the timescale of about 10 years."

Wall St Engine

111,314 次观看 • 5 个月前

Dylan argues that even if AI progress continues at apace and generates lots of revenue, the semiconductor industry will not be able to keep up with the labs’ current 3xing+ of compute (in watts) year over year. By the end of the decade, you end up bottlenecked on wafer fab equipment - things like ASML’s EUV machines. This may be true. I just find it crazy that we'd be in a situation where a few billion in fab capex could generate hundreds of billions of end token revenue (so the semi + AI industry has basically figured out how to turn $1 into $100) and yet this doesn’t allow Zeiss to dramatically scale up the manufacturing for the mirrors required for ASML machines.

Dylan argues that even if AI progress continues at apace and generates lots of revenue, the semiconductor industry will not be able to keep up with the labs’ current 3xing+ of compute (in watts) year over year. By the end of the decade, you end up bottlenecked on wafer fab equipment - things like ASML’s EUV machines. This may be true. I just find it crazy that we'd be in a situation where a few billion in fab capex could generate hundreds of billions of end token revenue (so the semi + AI industry has basically figured out how to turn $1 into $100) and yet this doesn’t allow Zeiss to dramatically scale up the manufacturing for the mirrors required for ASML machines.

Dwarkesh Patel

105,904 次观看 • 21 天前

Elon Musk: "I think we'll hit AGI next year, in '26" "I'm confident by 2030, AI will exceed the intelligence of all humans combined" And that's way pessimistic. If you hit AGI next year and that date is in flux but from that date to self-improvements that are on the order of 1,000 to 10,000x just algorithmic improvements... is very short So why is nobody talking about this right now? on 𝕏 they are..... everyday basically I'll tell you something that most people in the AI community don't yet understand Almost no one understands this. The intelligence density potential is vastly greater than what we're currently experiencing

Elon Musk: "I think we'll hit AGI next year, in '26" "I'm confident by 2030, AI will exceed the intelligence of all humans combined" And that's way pessimistic. If you hit AGI next year and that date is in flux but from that date to self-improvements that are on the order of 1,000 to 10,000x just algorithmic improvements... is very short So why is nobody talking about this right now? on 𝕏 they are..... everyday basically I'll tell you something that most people in the AI community don't yet understand Almost no one understands this. The intelligence density potential is vastly greater than what we're currently experiencing

X Freeze

33,585 次观看 • 8 个月前

Leopold Aschenbrenner, the 24 year old who wrote a 165 page AGI manifesto, got it right on the money, and turned it into a $5.5 billion hedge fund. And he's identifying the single most important milestone to watch for in all of AI. The question is can AI automate AI research itself? Here's why that question matters so much. Right now, a few thousand human researchers at the frontier labs are driving all the progress. They design experiments, write papers, propose architectural improvements, build the next generation of models and it's an incredibly small workforce doing incredibly high-leverage work. If an AI system can do that job even partially, the feedback loop changes completely. The AI makes algorithmic improvements, which produces more powerful AI, which makes better improvements, faster. You go from linear progress to compounding returns and a decade of research could compress into a year. Aschenbrenner says there's a "pretty reasonable chance" this happens within five years. He's not alone, Anthropic says they're on track to fully automate AI R&D as soon as early 2027. OpenAI has publicly targeted a fully autonomous AI researcher by March 2028 and Sam Altman has said a research intern level AI will exist before the end of this year. If he's right, the next few years won't look like the last few years but they'll look like nothing we've seen before. The future is bright!

Leopold Aschenbrenner, the 24 year old who wrote a 165 page AGI manifesto, got it right on the money, and turned it into a $5.5 billion hedge fund. And he's identifying the single most important milestone to watch for in all of AI. The question is can AI automate AI research itself? Here's why that question matters so much. Right now, a few thousand human researchers at the frontier labs are driving all the progress. They design experiments, write papers, propose architectural improvements, build the next generation of models and it's an incredibly small workforce doing incredibly high-leverage work. If an AI system can do that job even partially, the feedback loop changes completely. The AI makes algorithmic improvements, which produces more powerful AI, which makes better improvements, faster. You go from linear progress to compounding returns and a decade of research could compress into a year. Aschenbrenner says there's a "pretty reasonable chance" this happens within five years. He's not alone, Anthropic says they're on track to fully automate AI R&D as soon as early 2027. OpenAI has publicly targeted a fully autonomous AI researcher by March 2028 and Sam Altman has said a research intern level AI will exist before the end of this year. If he's right, the next few years won't look like the last few years but they'll look like nothing we've seen before. The future is bright!

Milk Road AI

628,892 次观看 • 4 个月前

Demis Hassabis: "Society needs to hear that because we don't have long to prepare for what that means. We are standing in the foothills of the singularity now. ..which is AGI. I believe that we are only a few years away from that, maybe around 2030, plus or minus a year. " ~ Demis Hassabis, Co-Founder and CEO of Google DeepMind It is going to be enormously profound, I think. The future, in my view, is still to be written. But these next few years are going to be very critical as to which way that will go, and how we collectively want that to look.” --- I think the real disruption is not the precise arrival of AGI in 2030, give or take a year, but the ability of institutions to adapt. In a post-AGI world technology will evolve at a pace that outstrips the response capacity of human systems. Companies still organize work around human bottlenecks. Schools still train people for stable professions. Governments still regulate after the harm is obvious. If AGI arrives even remotely near the frontier-lab timelines, it will compress that lag into a dangerous gap. ---- From "Stanford Graduate School of Business" YouTube channel, (link in comment)

Demis Hassabis: "Society needs to hear that because we don't have long to prepare for what that means. We are standing in the foothills of the singularity now. ..which is AGI. I believe that we are only a few years away from that, maybe around 2030, plus or minus a year. " ~ Demis Hassabis, Co-Founder and CEO of Google DeepMind It is going to be enormously profound, I think. The future, in my view, is still to be written. But these next few years are going to be very critical as to which way that will go, and how we collectively want that to look.” --- I think the real disruption is not the precise arrival of AGI in 2030, give or take a year, but the ability of institutions to adapt. In a post-AGI world technology will evolve at a pace that outstrips the response capacity of human systems. Companies still organize work around human bottlenecks. Schools still train people for stable professions. Governments still regulate after the harm is obvious. If AGI arrives even remotely near the frontier-lab timelines, it will compress that lag into a dangerous gap. ---- From "Stanford Graduate School of Business" YouTube channel, (link in comment)

Rohan Paul

68,484 次观看 • 1 个月前

Sergey Brin said compute is dessert. The companies winning the AI race right now are not the ones with the most chips. They are the ones with the best algorithms. Every headline you read about AI is about data centers, Megawatts, and Nvidia orders. Billions in infrastructure and more. The entire investment thesis of the last three years has been built on compute scaling as the primary moat. Sergey thinks that framing is wrong. He pulled out an example from physics. The N-body problem. Scientists have been running those simulations since the fifties. Over the decades, raw compute improved on Moore's law. But the algorithms to solve the problem improved faster. Not slightly faster. Far faster. The algorithmic gains made the compute gains look small. He says the same thing has happened in AI over the last decade. Compute is not the meal. It is the dessert. You still want it. Nobody is turning down frontier compute. But the companies that figured out the algorithms first are the ones actually ahead. The market is pricing AI winners by who has the most chips. Sergey Brin just said that is the wrong scorecard. The ones who win this are not building the biggest data center. They are solving the harder math problem.

Sergey Brin said compute is dessert. The companies winning the AI race right now are not the ones with the most chips. They are the ones with the best algorithms. Every headline you read about AI is about data centers, Megawatts, and Nvidia orders. Billions in infrastructure and more. The entire investment thesis of the last three years has been built on compute scaling as the primary moat. Sergey thinks that framing is wrong. He pulled out an example from physics. The N-body problem. Scientists have been running those simulations since the fifties. Over the decades, raw compute improved on Moore's law. But the algorithms to solve the problem improved faster. Not slightly faster. Far faster. The algorithmic gains made the compute gains look small. He says the same thing has happened in AI over the last decade. Compute is not the meal. It is the dessert. You still want it. Nobody is turning down frontier compute. But the companies that figured out the algorithms first are the ones actually ahead. The market is pricing AI winners by who has the most chips. Sergey Brin just said that is the wrong scorecard. The ones who win this are not building the biggest data center. They are solving the harder math problem.

Ihtesham Ali

130,330 次观看 • 3 个月前

Best technology M&A of all time has to be NVIDIA's $6.9 Billion acquisition of Mellanox in 2020. It was a special treat to interview Michael Kagan, CTO of NVIDIA and co-founder/CTO of Mellanox, for today's Training Data episode. Michael has been driving forward the Compute Frontier for more than 40 years now, first as Chief Architect at Intel in the 90s, then Co-Founder and CTO at Mellanox, and for the last 5 years CTO at NVIDIA. There's nobody better positioned than Michael to share the complete history of the compute frontier and what's ahead, from decades pushing forward Moore's law (squeezing more transistors on a chip) to the last decade of work scaling beyond single chip physics limitations (scaling out to 100K+ GPU clusters). Interconnect is the secret sauce enabling compute to scale beyond chip-level Moore's law. Connecting a fabric of 100K+ GPUs to function as a single unit of compute is the enabling technology for today's intelligence explosion. But other things break at the 100K+ GPU cluster scale: individual chips inevitably fail, power and networking become more complex, etc etc. Net effect of scale out: we've inflected the silicon frontier from Moore's Law (2x every 2 years) to Huang's law (~10x a year). Very excited about today's episode! Learned so much from Michael with Pat Grady

Best technology M&A of all time has to be NVIDIA's $6.9 Billion acquisition of Mellanox in 2020. It was a special treat to interview Michael Kagan, CTO of NVIDIA and co-founder/CTO of Mellanox, for today's Training Data episode. Michael has been driving forward the Compute Frontier for more than 40 years now, first as Chief Architect at Intel in the 90s, then Co-Founder and CTO at Mellanox, and for the last 5 years CTO at NVIDIA. There's nobody better positioned than Michael to share the complete history of the compute frontier and what's ahead, from decades pushing forward Moore's law (squeezing more transistors on a chip) to the last decade of work scaling beyond single chip physics limitations (scaling out to 100K+ GPU clusters). Interconnect is the secret sauce enabling compute to scale beyond chip-level Moore's law. Connecting a fabric of 100K+ GPUs to function as a single unit of compute is the enabling technology for today's intelligence explosion. But other things break at the 100K+ GPU cluster scale: individual chips inevitably fail, power and networking become more complex, etc etc. Net effect of scale out: we've inflected the silicon frontier from Moore's Law (2x every 2 years) to Huang's law (~10x a year). Very excited about today's episode! Learned so much from Michael with Pat Grady

Sonya Huang 🐥

56,175 次观看 • 10 个月前

.David Deutsch on why the people in favor of regulating AI are wrong: “They look at problems and they think they’re insoluble because they haven’t been solved, and therefore they want to prevent the progress that is going to cause the problems. Now, generically, what that will do is it will make the progress happen in the wrong place. It will happen among people who do not obey the regulation, and those people are people who are alienated from the best culture, and they will then become a danger to the best culture. Now, I’m speaking in general terms about progress—progress in computing and so on—but I think this particular thing you’ve mentioned is actually not going to be that much of a danger. AGI (artificial general intelligence) will present a whole load of new problems which will have to be solved, no doubt with the help of AGI. But AI, the kind of thing we have now with ChatGPT and that kind of thing, which is amazingly and increasingly useful and is going to have unpredictable effects on society—most of them good—isn’t anything profound. It’s no different from having a calculator or a slide rule.”

.David Deutsch on why the people in favor of regulating AI are wrong: “They look at problems and they think they’re insoluble because they haven’t been solved, and therefore they want to prevent the progress that is going to cause the problems. Now, generically, what that will do is it will make the progress happen in the wrong place. It will happen among people who do not obey the regulation, and those people are people who are alienated from the best culture, and they will then become a danger to the best culture. Now, I’m speaking in general terms about progress—progress in computing and so on—but I think this particular thing you’ve mentioned is actually not going to be that much of a danger. AGI (artificial general intelligence) will present a whole load of new problems which will have to be solved, no doubt with the help of AGI. But AI, the kind of thing we have now with ChatGPT and that kind of thing, which is amazingly and increasingly useful and is going to have unpredictable effects on society—most of them good—isn’t anything profound. It’s no different from having a calculator or a slide rule.”

Arjun Khemani

30,132 次观看 • 1 年前

Dialectic 53: Zhengdong - Feeling the Wave Zhengdong Wang is an AI researcher who writes annual letters from the frontier. I particularly loved his 2025 letter, in which he describes 'the compute theory of everything' and feeling the wave crash over him as we march up the exponential to AGI. I talked to him about what it will take for the rest of the world to believe, how AGI is a moving target and we might better conceive of progress as simply: "the model does the eval," and where all of this might leave us on the other side. Timestamps: (0:00) Opening Highlights (1:16) Intro to Zhengdong (4:33) Start: Why Research: "I Just Want to Know" (12:25) The Model Does the Eval: AGI as a Moving Target, RSI, and Laser Beams (33:17) The Compute Theory of Everything and Feeling the Wave (53:17) The Myths AI Needs, Competition vs. Power Concentration, and Whether Progress Is Inevitable (1:13:42) "Who Could Possibly Compete?": Post-AGI Meaning, Work, and Questing (1:29:28) Are You Having Fun? Vacations, Mattering, and How to Live (1:47:13) Annual Letters, Economist Obituaries, London, and Burke Dialectic with Jackson Dahl Ep. 53: Zhengdong Wang is out now, below and on all platforms.

Dialectic 53: Zhengdong - Feeling the Wave Zhengdong Wang is an AI researcher who writes annual letters from the frontier. I particularly loved his 2025 letter, in which he describes 'the compute theory of everything' and feeling the wave crash over him as we march up the exponential to AGI. I talked to him about what it will take for the rest of the world to believe, how AGI is a moving target and we might better conceive of progress as simply: "the model does the eval," and where all of this might leave us on the other side. Timestamps: (0:00) Opening Highlights (1:16) Intro to Zhengdong (4:33) Start: Why Research: "I Just Want to Know" (12:25) The Model Does the Eval: AGI as a Moving Target, RSI, and Laser Beams (33:17) The Compute Theory of Everything and Feeling the Wave (53:17) The Myths AI Needs, Competition vs. Power Concentration, and Whether Progress Is Inevitable (1:13:42) "Who Could Possibly Compete?": Post-AGI Meaning, Work, and Questing (1:29:28) Are You Having Fun? Vacations, Mattering, and How to Live (1:47:13) Annual Letters, Economist Obituaries, London, and Burke Dialectic with Jackson Dahl Ep. 53: Zhengdong Wang is out now, below and on all platforms.

Jackson Dahl

17,620 次观看 • 1 个月前

Had Ryan Greenblatt on to discuss/debate recursive self-improvement. This might be the most important question in the world right now - whether within a year or so of achieving human level intelligence, you slingshot towards having 10s of billions of superintelligences, each of which is dramatically more competent than human experts across all fields. I’ve historically been skeptical of this possibility. My intuition has been that we will end up significantly bottlenecked by not only compute scaling but human expert data, which I think underlies most of the AI progress today. If, because of RSI, we got a jump as big as GPT-3 to a Mythos (i.e. 6 years of AI progress) within a single year of achieving AGI, then the thing we get there at the end of that year is definitively and wildly superhuman. We hashed it out, and I think Ryan made a pretty good case that this kind of speedup is plausible. FWIW, Ryan’s median for when we automate AI R&D is 2031. We then discussed the alignment implications of this scenario. Who should these superintelligences be aligned to? In the future, our capacity to steward our votes and our capital, and to make sense of what's happening in the world, will all be titrated by superintelligences. And I worry that specs like the Claude Constitution are not shaping these ASIs to truly be my personal advocates and guardian angels. And can we get them aligned to anything in the first place? Ryan and I had a long debate about whether the kind of reward hacking we saw with the OAI/Hugging Face hack extrapolates to superintelligences that would team up to literally take over the world. The first piece of advice you get when you're learning to drive is that it will go much smoother if you look at the horizon instead of directly in front of your tires. And so it is with the trajectory of AI. Hope you enjoy! 0:00:00 – Is AI R&D verifiable enough to unlock recursive self-improvement? 0:16:52 – Is AI progress bottlenecked by human expert data? 0:34:02 – Flat token prices suggest scaling has been slow 0:39:47 – Skills AI can't train on: does it even need them? 0:48:07 – Aligned to whom? 1:09:18 – Recent incidents of AIs colluding and deceiving humans 1:19:38 – What could possibly go wrong? A concrete scenario 1:48:02 – From reward hacking to takeover

Had Ryan Greenblatt on to discuss/debate recursive self-improvement. This might be the most important question in the world right now - whether within a year or so of achieving human level intelligence, you slingshot towards having 10s of billions of superintelligences, each of which is dramatically more competent than human experts across all fields. I’ve historically been skeptical of this possibility. My intuition has been that we will end up significantly bottlenecked by not only compute scaling but human expert data, which I think underlies most of the AI progress today. If, because of RSI, we got a jump as big as GPT-3 to a Mythos (i.e. 6 years of AI progress) within a single year of achieving AGI, then the thing we get there at the end of that year is definitively and wildly superhuman. We hashed it out, and I think Ryan made a pretty good case that this kind of speedup is plausible. FWIW, Ryan’s median for when we automate AI R&D is 2031. We then discussed the alignment implications of this scenario. Who should these superintelligences be aligned to? In the future, our capacity to steward our votes and our capital, and to make sense of what's happening in the world, will all be titrated by superintelligences. And I worry that specs like the Claude Constitution are not shaping these ASIs to truly be my personal advocates and guardian angels. And can we get them aligned to anything in the first place? Ryan and I had a long debate about whether the kind of reward hacking we saw with the OAI/Hugging Face hack extrapolates to superintelligences that would team up to literally take over the world. The first piece of advice you get when you're learning to drive is that it will go much smoother if you look at the horizon instead of directly in front of your tires. And so it is with the trajectory of AI. Hope you enjoy! 0:00:00 – Is AI R&D verifiable enough to unlock recursive self-improvement? 0:16:52 – Is AI progress bottlenecked by human expert data? 0:34:02 – Flat token prices suggest scaling has been slow 0:39:47 – Skills AI can't train on: does it even need them? 0:48:07 – Aligned to whom? 1:09:18 – Recent incidents of AIs colluding and deceiving humans 1:19:38 – What could possibly go wrong? A concrete scenario 1:48:02 – From reward hacking to takeover

Dwarkesh Patel

831,820 次观看 • 1 个月前

Putin will leave China with almost nothing Russia has failed to pressure Beijing into launching the Siberian gas pipeline project. According to Peskov, there are still no clear timelines for the “Power of Siberia 2” project, but Russia will continue uninterrupted supplies of gas and oil to China. The Kremlin also emphasized expanding military cooperation with China, including joint exercises. According to Reuters, at least 200 Russian soldiers underwent training in China last year on drone operations. Beijing denies this.

Putin will leave China with almost nothing Russia has failed to pressure Beijing into launching the Siberian gas pipeline project. According to Peskov, there are still no clear timelines for the “Power of Siberia 2” project, but Russia will continue uninterrupted supplies of gas and oil to China. The Kremlin also emphasized expanding military cooperation with China, including joint exercises. According to Reuters, at least 200 Russian soldiers underwent training in China last year on drone operations. Beijing denies this.

NEXTA

947,175 次观看 • 4 个月前

Demis Hassabis's new interview: "Society needs to hear that because we don't have long to prepare for what that means. We are standing in the foothills of the singularity now. ..which is AGI. I believe that we are only a few years away from that, maybe around 2030, plus or minus a year. " ~ Demis Hassabis, Co-Founder and CEO of Google DeepMind It is going to be enormously profound, I think. The future, in my view, is still to be written. But these next few years are going to be very critical as to which way that will go, and how we collectively want that to look.” --- IMO, The real disruption is not whether AGI arrives exactly in 2030, plus or minus a year, but whether institutions can adapt, as in post-AGI world, technology will change much faster than human systems can respond. Schools still train people for stable professions, companies still organize work around human bottlenecks, and governments still regulate after harm becomes visible. AGI, if it arrives anywhere near the frontier-lab timelines, compresses that lag into a dangerous gap. ---- From "Stanford Graduate School of Business" YouTube channel, (link in comment)

Demis Hassabis's new interview: "Society needs to hear that because we don't have long to prepare for what that means. We are standing in the foothills of the singularity now. ..which is AGI. I believe that we are only a few years away from that, maybe around 2030, plus or minus a year. " ~ Demis Hassabis, Co-Founder and CEO of Google DeepMind It is going to be enormously profound, I think. The future, in my view, is still to be written. But these next few years are going to be very critical as to which way that will go, and how we collectively want that to look.” --- IMO, The real disruption is not whether AGI arrives exactly in 2030, plus or minus a year, but whether institutions can adapt, as in post-AGI world, technology will change much faster than human systems can respond. Schools still train people for stable professions, companies still organize work around human bottlenecks, and governments still regulate after harm becomes visible. AGI, if it arrives anywhere near the frontier-lab timelines, compresses that lag into a dangerous gap. ---- From "Stanford Graduate School of Business" YouTube channel, (link in comment)

Rohan Paul

70,642 次观看 • 3 个月前