Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Every frontier lab's stated plan for AGI 'crunch-time' is 'use AI to make AI safe.' I spoke with Ajeya Cotra – influential AI forecaster – who has been trying to figure out whether this crazy-sounding plan could actually work, and if so how: • Ajeya’s impressive track record identifying...

robertwiblin's profile picture

Rob Wiblin

50,908 subscribers

40,746 görüntüleme • 5 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

My interview with Helen Toner on firing Altman, AGI geopolitics in China & Middle East, and military use of AI. Toner co-founded and now leads the Center for Security and Emerging Technology at Georgetown in DC, maybe the leading think tank on natsec and AI. We discuss: • Is China actually racing to build AGI? [00:27:56] • Why you shouldn't say 'alignment' [01:45:11] • Is Congress starting to take superintelligence seriously? [01:47:42] • Are data centres in the UAE “good for democracy”? [00:52:41] • Sure China could steal AI model weights, but does it matter [00:39:13] • Is it good if the world uses US not Chinese AI models? [00:21:58] • Why non-proliferation is a misguided model for AI [01:30:20] • Helen thinks AI progress isn't slowing down [01:50:10] • Will AI inevitably concentrate power? [01:07:49] • Helen’s defence of how the board handled firing Altman [00:01:32] • Will the military be willing to defer to AI in decision-making? [01:38:23] Available any place you get podcasts on the '80,000 Hours Podcast'. Links below — enjoy!
2:23:01

My interview with Helen Toner on firing Altman, AGI geopolitics in China & Middle East, and military use of AI. Toner co-founded and now leads the Center for Security and Emerging Technology at Georgetown in DC, maybe the leading think tank on natsec and AI. We discuss: • Is China actually racing to build AGI? [00:27:56] • Why you shouldn't say 'alignment' [01:45:11] • Is Congress starting to take superintelligence seriously? [01:47:42] • Are data centres in the UAE “good for democracy”? [00:52:41] • Sure China could steal AI model weights, but does it matter [00:39:13] • Is it good if the world uses US not Chinese AI models? [00:21:58] • Why non-proliferation is a misguided model for AI [01:30:20] • Helen thinks AI progress isn't slowing down [01:50:10] • Will AI inevitably concentrate power? [01:07:49] • Helen’s defence of how the board handled firing Altman [00:01:32] • Will the military be willing to defer to AI in decision-making? [01:38:23] Available any place you get podcasts on the '80,000 Hours Podcast'. Links below — enjoy!

Rob Wiblin

20,369 görüntüleme • 9 ay önce

Could AI deserve rights before it even feels like anything to be an AI? Oxford moral philosopher Andreas Mogensen thinks... maybe. In her first interview for the 80,000 Hours Podcast my colleague Zershaaneh Qureshi pushes Andreas on that, as well as new arguments for why human extinction could be a good thing: The puzzle of moral patienthood (00:03:20) Is subjective experience necessary? (00:05:52) What is it to desire? (00:10:42) Desiring without experiencing (00:17:56) What would make AIs moral patients? (00:28:17) Another route entirely: deserving autonomy (00:45:12) Maybe there's no objective truth about any of this (01:12:06) Practical implications (01:29:21) Why not just let superintelligence figure this out for us? (01:38:07) How could human extinction be a good thing? (01:47:30) Lexical threshold negative utilitarianism (02:12:30) So... should we still try to prevent extinction? (02:25:22) What are the most important questions for people to address here? (02:32:16) Is God GDPR compliant? (02:35:32) Links below, enjoy!
2:37:14

Could AI deserve rights before it even feels like anything to be an AI? Oxford moral philosopher Andreas Mogensen thinks... maybe. In her first interview for the 80,000 Hours Podcast my colleague Zershaaneh Qureshi pushes Andreas on that, as well as new arguments for why human extinction could be a good thing: The puzzle of moral patienthood (00:03:20) Is subjective experience necessary? (00:05:52) What is it to desire? (00:10:42) Desiring without experiencing (00:17:56) What would make AIs moral patients? (00:28:17) Another route entirely: deserving autonomy (00:45:12) Maybe there's no objective truth about any of this (01:12:06) Practical implications (01:29:21) Why not just let superintelligence figure this out for us? (01:38:07) How could human extinction be a good thing? (01:47:30) Lexical threshold negative utilitarianism (02:12:30) So... should we still try to prevent extinction? (02:25:22) What are the most important questions for people to address here? (02:32:16) Is God GDPR compliant? (02:35:32) Links below, enjoy!

Rob Wiblin

10,720 görüntüleme • 7 ay önce

I interviewed Holden Karnofsky of Anthropic (and past CEO of Open Philanthropy) for 4.5 hours on almost all his AGI takes: • The AI 'race' isn't a coordination failure [00:18:01] • Why he's currently focused on pursuing easy wins and partial victories [02:43:43] • "It'll be the second advanced species ever" [02:31:44] • Dozens of 'concrete shovel-ready projects' he's excited about [01:17:58] • People don't appreciate all the engaging, impactful, high-feedback work available tackling AGI risks [01:19:00] • Having a more responsible AI company really does matter [02:43:43] • You shouldn't count on trusting anyone in AI [00:44:30] • To take over an AGI might just wait it out [00:07:39] • Farm animal welfare campaigns that targeted companies contain some useful lessons here [02:34:18] • Human-AI relationships really are troubling [03:53:58] • It's totally plausible we incompetently fumble our way into a great future [03:04:16)] • AI R&D is *the* thing to worry about [01:57:31] • And many others. Holden has opinions on almost everything and isn't afraid to speak his mind. Enjoy! (On the 80,000 Hours Podcast anywhere you get podcasts. Links below.)
4:34:57

I interviewed Holden Karnofsky of Anthropic (and past CEO of Open Philanthropy) for 4.5 hours on almost all his AGI takes: • The AI 'race' isn't a coordination failure [00:18:01] • Why he's currently focused on pursuing easy wins and partial victories [02:43:43] • "It'll be the second advanced species ever" [02:31:44] • Dozens of 'concrete shovel-ready projects' he's excited about [01:17:58] • People don't appreciate all the engaging, impactful, high-feedback work available tackling AGI risks [01:19:00] • Having a more responsible AI company really does matter [02:43:43] • You shouldn't count on trusting anyone in AI [00:44:30] • To take over an AGI might just wait it out [00:07:39] • Farm animal welfare campaigns that targeted companies contain some useful lessons here [02:34:18] • Human-AI relationships really are troubling [03:53:58] • It's totally plausible we incompetently fumble our way into a great future [03:04:16)] • AI R&D is *the* thing to worry about [01:57:31] • And many others. Holden has opinions on almost everything and isn't afraid to speak his mind. Enjoy! (On the 80,000 Hours Podcast anywhere you get podcasts. Links below.)

Rob Wiblin

73,586 görüntüleme • 9 ay önce

Yoshua Bengio thinks he knows how to make provably safe superintelligent agents. Bengio built the foundations of modern AI and is the most cited living scientist. He believes his alternative training setup would: 1. Guarantee honesty 2. Prevent unintended goals 3. Produce capable agents 4. Port over most data and techniques from current LLMs 5. Not be inherently more expensive, and perhaps be more intelligent Bengio claims the honesty and lack of unintended goals can be proven mathematically, at least given particular assumptions. And his new organization, LawZero, is aiming to build a scrappy prototype as soon as possible. The architecture is called 'Scientist AI' and it's based on training a model to explain empirical observations, including what people say, rather than training AIs that mimic human behaviour or seek our approval. (Bengio's frank assessment is that "reinforcement learning is evil" and that allowing AIs to independently train their successors is "the most crazy, dangerous bet that unfortunately we are on track to do.") But skeptics question whether Scientist AI really does solve the fundamental problem of 'eliciting latent knowledge' from AI models. And with the commercial race for superintelligence so intense, it's not clear whether the proposal will be able to compete or have time to bear fruit, even if it's sound in theory. On The 80,000 Hours Podcast, links below – enjoy! • Making AI honest and safe (00:00:00) • Scientist AI in plain English (00:02:27) • How Scientist AI differs from LLMs (00:06:32) • How the training data works (00:14:02) • Can this become an agent? (00:21:02) • Why Yoshua is now more optimistic (00:32:11) • Why companies can’t stop racing (00:36:35) • A working prototype won't take long (00:49:15) • Scientist models might be more capable (00:53:34) • “Reinforcement learning is evil” (01:01:27) • Scientist AI from guardrail to agent (01:08:37) • Can safe AI still be competent? (01:12:38) • How much will this cost? (01:19:29) • Can it generalise beyond maths and science? (01:23:26) • A multi-national push for superintelligence (01:39:19) • Want to work with or fund Yoshua? (01:51:16) • Why smart people ignore AI risk (01:54:45) • Don’t let AI build the next AI (02:01:33) • Why politicians miss the real risks (02:12:28) • Why Yoshua changed his mind about AI risk (02:21:27)
2:35:26

Yoshua Bengio thinks he knows how to make provably safe superintelligent agents. Bengio built the foundations of modern AI and is the most cited living scientist. He believes his alternative training setup would: 1. Guarantee honesty 2. Prevent unintended goals 3. Produce capable agents 4. Port over most data and techniques from current LLMs 5. Not be inherently more expensive, and perhaps be more intelligent Bengio claims the honesty and lack of unintended goals can be proven mathematically, at least given particular assumptions. And his new organization, LawZero, is aiming to build a scrappy prototype as soon as possible. The architecture is called 'Scientist AI' and it's based on training a model to explain empirical observations, including what people say, rather than training AIs that mimic human behaviour or seek our approval. (Bengio's frank assessment is that "reinforcement learning is evil" and that allowing AIs to independently train their successors is "the most crazy, dangerous bet that unfortunately we are on track to do.") But skeptics question whether Scientist AI really does solve the fundamental problem of 'eliciting latent knowledge' from AI models. And with the commercial race for superintelligence so intense, it's not clear whether the proposal will be able to compete or have time to bear fruit, even if it's sound in theory. On The 80,000 Hours Podcast, links below – enjoy! • Making AI honest and safe (00:00:00) • Scientist AI in plain English (00:02:27) • How Scientist AI differs from LLMs (00:06:32) • How the training data works (00:14:02) • Can this become an agent? (00:21:02) • Why Yoshua is now more optimistic (00:32:11) • Why companies can’t stop racing (00:36:35) • A working prototype won't take long (00:49:15) • Scientist models might be more capable (00:53:34) • “Reinforcement learning is evil” (01:01:27) • Scientist AI from guardrail to agent (01:08:37) • Can safe AI still be competent? (01:12:38) • How much will this cost? (01:19:29) • Can it generalise beyond maths and science? (01:23:26) • A multi-national push for superintelligence (01:39:19) • Want to work with or fund Yoshua? (01:51:16) • Why smart people ignore AI risk (01:54:45) • Don’t let AI build the next AI (02:01:33) • Why politicians miss the real risks (02:12:28) • Why Yoshua changed his mind about AI risk (02:21:27)

Rob Wiblin

65,088 görüntüleme • 3 ay önce

How does math research change when the cost of trying your first dumb idea goes to zero? Daniel Litt joins Greg Burnham and Anson Ho to discuss what today’s models can and can’t do in math, and how far they are from doing high-quality research. 0:00:00 What's the hardest math problem AI can solve today? 00:16:08 How helpful are today’s AI models for math research? 00:23:36 Junk papers, LLM-generated proofs, and the refereeing crisis 00:27:21 AI enables searching through problems at scale 00:33:49 When will AI be good enough to publish in top math journals? 00:42:15 What are the returns to intelligence? 00:59:50 Will AI solve Millennium problems? 01:11:54 Is math full of low-hanging fruit? 01:18:47 How Daniel has adapted his professional life to AI progress 01:25:28 What do AI math benchmarks actually measure? 01:33:05 Designing the Open Problems benchmark 01:56:35 Do mathematicians believe heuristic arguments about conjectures? 02:01:24 What if FrontierMath: Open Problems gets solved? 02:06:53 Is AI on the cusp of accelerating math progress?
2:10:59

How does math research change when the cost of trying your first dumb idea goes to zero? Daniel Litt joins Greg Burnham and Anson Ho to discuss what today’s models can and can’t do in math, and how far they are from doing high-quality research. 0:00:00 What's the hardest math problem AI can solve today? 00:16:08 How helpful are today’s AI models for math research? 00:23:36 Junk papers, LLM-generated proofs, and the refereeing crisis 00:27:21 AI enables searching through problems at scale 00:33:49 When will AI be good enough to publish in top math journals? 00:42:15 What are the returns to intelligence? 00:59:50 Will AI solve Millennium problems? 01:11:54 Is math full of low-hanging fruit? 01:18:47 How Daniel has adapted his professional life to AI progress 01:25:28 What do AI math benchmarks actually measure? 01:33:05 Designing the Open Problems benchmark 01:56:35 Do mathematicians believe heuristic arguments about conjectures? 02:01:24 What if FrontierMath: Open Problems gets solved? 02:06:53 Is AI on the cusp of accelerating math progress?

Epoch AI

178,703 görüntüleme • 6 ay önce

The Will MacAskill episode on: 1. Why we should both pay AIs and trade with them (00:38:13) 2. Why not just push to pause AI development? (01:42:17) 3. Making AIs extremely risk averse might also make them safe (00:38:13) 4. The panic over sycophancy was justified (08:11) 5. The character we give AI models could determine the future (00:01:00) 6. The toughest trade-offs in AI personality design (13:24) 7. The effective altruism comeback (01:52:19) 8. Every population ethics recommends 'tiling the universe', except Will's new one (02:39:35) 9. Non-causal decision theory might accidentally motivate a near-best future (01:22:19) 10. How to aim at the best possible world without utopianism (02:09:30) For the 80,000 Hours Podcast, links below - enjoy! William MacAskill
3:14:53

The Will MacAskill episode on: 1. Why we should both pay AIs and trade with them (00:38:13) 2. Why not just push to pause AI development? (01:42:17) 3. Making AIs extremely risk averse might also make them safe (00:38:13) 4. The panic over sycophancy was justified (08:11) 5. The character we give AI models could determine the future (00:01:00) 6. The toughest trade-offs in AI personality design (13:24) 7. The effective altruism comeback (01:52:19) 8. Every population ethics recommends 'tiling the universe', except Will's new one (02:39:35) 9. Non-causal decision theory might accidentally motivate a near-best future (01:22:19) 10. How to aim at the best possible world without utopianism (02:09:30) For the 80,000 Hours Podcast, links below - enjoy! William MacAskill

Rob Wiblin

40,908 görüntüleme • 3 ay önce

My conversation with OpenAI co-founder Greg Brockman This is the most detailed first-person account of the 72 hours after Sam Altman was fired. We also go deep on what comes next: the global race to AGI, why ChatGPT stopped showing reasoning, how much of OpenAI's own code is now written by AI ("it's hard to know what percent is not"), and the untold story of how OpenAI actually started in 2015. 00:00:00 Introduction 00:00:49 Meeting Sam Altman and Starting OpenAI 00:02:40 Building the Founding Team 00:04:25 DeepMind's Lead Over OpenAI 00:04:54 Changing OpenAI to a For-Profit Model 00:06:05 Breakthrough Moments at OpenAI 00:08:22 What Dota 2 Meant for OpenAI 00:10:04 Reasoning Versus Prediction 00:11:59 Tensions Grow at OpenAI 00:15:44 Sam Altman's Firing 00:17:49 Greg Quits OpenAI 00:19:56 Sam Explores Deal with Microsoft's Satya 00:20:28 Petition for Altman's Return 00:23:43 Ilya Sutskever Leaves OpenAI 00:24:59 Lessons Learned after Sam Ousting 00:28:22 The Thing Ilya Said that Greg Can't Forget 00:32:22 Is AI Going Parabolic? 00:33:24 How Much of OpenAI's Code is Written by AI? 00:36:21 Do AI Chatbots Tell Us What We Want to Hear? 00:38:06 The Global AI Race to Reach AGI 00:38:40 What Happens if US Doesn't Reach AGI First? 00:39:49 Are Countries Stealing AI Advancements? 00:40:38 Why ChatGPT No Longer Shows Reasoning 00:41:47 The Finite Constraints of Compute 00:43:38 On Investing Early in Data Centers 00:46:31 The Future of Data Center Specialization 00:47:52 How to Decide Whose Queries to Serve 00:49:08 OpenAI on Consumer vs Enterprise Models 00:53:05 Data Centers in Space? 01:00:56 What Should AI Regulation Look Like? 01:04:33 The Future of AI-Powered Entrepreneurship 01:04:44 AI and Job Loss 01:07:15 The Skills Young People Should Invest In 01:11:30 What Does Success Look Like For You? Full episode on X below. Also find it on: • YouTube: • Spotify: • Apple:
1:11:41

My conversation with OpenAI co-founder Greg Brockman This is the most detailed first-person account of the 72 hours after Sam Altman was fired. We also go deep on what comes next: the global race to AGI, why ChatGPT stopped showing reasoning, how much of OpenAI's own code is now written by AI ("it's hard to know what percent is not"), and the untold story of how OpenAI actually started in 2015. 00:00:00 Introduction 00:00:49 Meeting Sam Altman and Starting OpenAI 00:02:40 Building the Founding Team 00:04:25 DeepMind's Lead Over OpenAI 00:04:54 Changing OpenAI to a For-Profit Model 00:06:05 Breakthrough Moments at OpenAI 00:08:22 What Dota 2 Meant for OpenAI 00:10:04 Reasoning Versus Prediction 00:11:59 Tensions Grow at OpenAI 00:15:44 Sam Altman's Firing 00:17:49 Greg Quits OpenAI 00:19:56 Sam Explores Deal with Microsoft's Satya 00:20:28 Petition for Altman's Return 00:23:43 Ilya Sutskever Leaves OpenAI 00:24:59 Lessons Learned after Sam Ousting 00:28:22 The Thing Ilya Said that Greg Can't Forget 00:32:22 Is AI Going Parabolic? 00:33:24 How Much of OpenAI's Code is Written by AI? 00:36:21 Do AI Chatbots Tell Us What We Want to Hear? 00:38:06 The Global AI Race to Reach AGI 00:38:40 What Happens if US Doesn't Reach AGI First? 00:39:49 Are Countries Stealing AI Advancements? 00:40:38 Why ChatGPT No Longer Shows Reasoning 00:41:47 The Finite Constraints of Compute 00:43:38 On Investing Early in Data Centers 00:46:31 The Future of Data Center Specialization 00:47:52 How to Decide Whose Queries to Serve 00:49:08 OpenAI on Consumer vs Enterprise Models 00:53:05 Data Centers in Space? 01:00:56 What Should AI Regulation Look Like? 01:04:33 The Future of AI-Powered Entrepreneurship 01:04:44 AI and Job Loss 01:07:15 The Skills Young People Should Invest In 01:11:30 What Does Success Look Like For You? Full episode on X below. Also find it on: • YouTube: • Spotify: • Apple:

Shane Parrish

450,952 görüntüleme • 3 ay önce

Here is my episode with Patrick Collison We discuss: - what it takes to process $1 trillion/year - how to build multi-decade APIs, companies, and relationships - what's next for Stripe (increasing the GDP of the internet is an open ended task, and the Collison brothers are just getting started). Plus the amazing stuff they're doing at Arc Institute, the financial infrastructure for AI agents, playing devil's advocate on progress studies, and much more. Links to other platforms below. Enjoy! Timestamps: 00:00:46 - Advice for 20-30 year olds 00:12:58 - Progress studies 00:23:07 - Arc Institute 00:35:13 - AI & Fast Grants 00:44:32 - Stripe history 00:56:30 - Stripe Climate 01:02:25 - Beauty & APIs 01:12:37 - Financial innards 01:29:02 - Stripe culture & future 01:42:42 - Virtues of big businesses 01:52:27 - John
1:57:38

Here is my episode with Patrick Collison We discuss: - what it takes to process $1 trillion/year - how to build multi-decade APIs, companies, and relationships - what's next for Stripe (increasing the GDP of the internet is an open ended task, and the Collison brothers are just getting started). Plus the amazing stuff they're doing at Arc Institute, the financial infrastructure for AI agents, playing devil's advocate on progress studies, and much more. Links to other platforms below. Enjoy! Timestamps: 00:00:46 - Advice for 20-30 year olds 00:12:58 - Progress studies 00:23:07 - Arc Institute 00:35:13 - AI & Fast Grants 00:44:32 - Stripe history 00:56:30 - Stripe Climate 01:02:25 - Beauty & APIs 01:12:37 - Financial innards 01:29:02 - Stripe culture & future 01:42:42 - Virtues of big businesses 01:52:27 - John

Dwarkesh Patel

350,541 görüntüleme • 2 yıl önce

Netflix pays AI PMs up to $900K/year. How do you become (and succeed) as one? Marily Nika Marily Nika is an AI PM at Google, and was an AI PM at Meta. In today's episode, she breaks it down: How AI PM Roles Differ at Big Tech Companies – 06:56 How Technical an AI PM Needs to Be – 08:08 Data Handling Responsibilities for AI PMs – 13:10 How PMs Can Improve Collaboration with Researchers – 18:06 Interviewing for AI PM Roles at Big Tech Companies – 22:08 The Role of a PhD in AI Product Management – 26:03 Optimizing LinkedIn for Aspiring AI PMs – 31:55 Roadmap to Becoming an AI PM – 37:05 Transitioning into AI Product Management – 40:52 The Process of Boomeranging Back to a Company – 44:00 AI Tools Every PM Should Know – 48:08 What It’s Like to Be an AI PM at Google – 57:46 How to Develop a Strong Product Strategy – 01:01:21 The Evolution of PM Roles in the Age of AI – 01:05:11 How PMs Can Stay Relevant in an AI-Driven World – 01:08:12 Balancing a Side Business with a Full-Time Job – 01:13:42 The Process of Creating a Course – 01:17:12 LinkedIn Tips for Aspiring AI PM Creators – 01:21:21 Lessons from Product Failures – 01:25:27 Paid promotions included
1:27:33

Netflix pays AI PMs up to $900K/year. How do you become (and succeed) as one? Marily Nika Marily Nika is an AI PM at Google, and was an AI PM at Meta. In today's episode, she breaks it down: How AI PM Roles Differ at Big Tech Companies – 06:56 How Technical an AI PM Needs to Be – 08:08 Data Handling Responsibilities for AI PMs – 13:10 How PMs Can Improve Collaboration with Researchers – 18:06 Interviewing for AI PM Roles at Big Tech Companies – 22:08 The Role of a PhD in AI Product Management – 26:03 Optimizing LinkedIn for Aspiring AI PMs – 31:55 Roadmap to Becoming an AI PM – 37:05 Transitioning into AI Product Management – 40:52 The Process of Boomeranging Back to a Company – 44:00 AI Tools Every PM Should Know – 48:08 What It’s Like to Be an AI PM at Google – 57:46 How to Develop a Strong Product Strategy – 01:01:21 The Evolution of PM Roles in the Age of AI – 01:05:11 How PMs Can Stay Relevant in an AI-Driven World – 01:08:12 Balancing a Side Business with a Full-Time Job – 01:13:42 The Process of Creating a Course – 01:17:12 LinkedIn Tips for Aspiring AI PM Creators – 01:21:21 Lessons from Product Failures – 01:25:27 Paid promotions included

Aakash Gupta

42,853 görüntüleme • 1 yıl önce

Every AI lab is working to make their AI helpful, harmless and honest. Max Harms (Max Harms) thinks this is a complete wrong turn, and 'aligning' AI to human values is actively dangerous. In his view a safe AGI must have absolutely no opinion about how the world ought to be, be willingly modifiable, and be entirely indifferent to being shut down. The opposite of all commercial models today. The key appeal is that so-called 'corrigibility' could be an attractor state – get close enough and the AI actively helps you make it more corrigible over time. That forgiveness would at least give us a shot. It's a strategy that feels natural within the 'MIRI worldview', recently laid out by his colleagues Eliezer Yudkowsky ⏹️ and Nate Soares ⏹️ in 'If Anyone Builds It Everyone Dies'. But it risks causing a different AI catastrophe, because the resulting AI model would necessarily be willing to assist any human operator with a power grab, or indeed any crime at all. I interviewed Max on the 80,000 Hours Podcast to debate the MIRI worldview, and what we should do to figure out if corrigibility ought to be our one and only focus. Links below – enjoy! 00:01:56 If anyone builds it, will everyone die? The MIRI perspective on AGI risk 00:24:28 Evolution failed to ‘align’ us, just as we'll fail to align AI 00:42:56 We're training AIs to want to stay alive and value power for its own sake 00:52:24 Objections: Is the 'squiggle/paperclip problem' really real? 01:05:02 Can we get empirical evidence re: 'alignment by default'? 01:10:17 Why do few AI researchers share Max's perspective? 01:18:34 We're training AI to pursue goals relentlessly — and superintelligence will too 01:24:51 The case for a radical slowdown 01:27:53 Max's best hope: corrigibility as stepping stone to alignment 01:32:34 Corrigibility is both uniquely valuable, and practical, to train 01:45:06 What training could ever make models corrigible enough? 01:51:38 Corrigibility is also terribly risky due to misuse risk 01:58:57 A single researcher could make a corrigibility benchmark. Nobody has. 02:12:20 Red Heart & why Max writes hard science fiction 02:34:08 Should you homeschool? Depends how weird your kids are.
2:40:21

Every AI lab is working to make their AI helpful, harmless and honest. Max Harms (Max Harms) thinks this is a complete wrong turn, and 'aligning' AI to human values is actively dangerous. In his view a safe AGI must have absolutely no opinion about how the world ought to be, be willingly modifiable, and be entirely indifferent to being shut down. The opposite of all commercial models today. The key appeal is that so-called 'corrigibility' could be an attractor state – get close enough and the AI actively helps you make it more corrigible over time. That forgiveness would at least give us a shot. It's a strategy that feels natural within the 'MIRI worldview', recently laid out by his colleagues Eliezer Yudkowsky ⏹️ and Nate Soares ⏹️ in 'If Anyone Builds It Everyone Dies'. But it risks causing a different AI catastrophe, because the resulting AI model would necessarily be willing to assist any human operator with a power grab, or indeed any crime at all. I interviewed Max on the 80,000 Hours Podcast to debate the MIRI worldview, and what we should do to figure out if corrigibility ought to be our one and only focus. Links below – enjoy! 00:01:56 If anyone builds it, will everyone die? The MIRI perspective on AGI risk 00:24:28 Evolution failed to ‘align’ us, just as we'll fail to align AI 00:42:56 We're training AIs to want to stay alive and value power for its own sake 00:52:24 Objections: Is the 'squiggle/paperclip problem' really real? 01:05:02 Can we get empirical evidence re: 'alignment by default'? 01:10:17 Why do few AI researchers share Max's perspective? 01:18:34 We're training AI to pursue goals relentlessly — and superintelligence will too 01:24:51 The case for a radical slowdown 01:27:53 Max's best hope: corrigibility as stepping stone to alignment 01:32:34 Corrigibility is both uniquely valuable, and practical, to train 01:45:06 What training could ever make models corrigible enough? 01:51:38 Corrigibility is also terribly risky due to misuse risk 01:58:57 A single researcher could make a corrigibility benchmark. Nobody has. 02:12:20 Red Heart & why Max writes hard science fiction 02:34:08 Should you homeschool? Depends how weird your kids are.

Rob Wiblin

296,553 görüntüleme • 5 ay önce

New podcast with Garry Tan, farbood and Daniel Francis. Live in the Future! 00:00 Guest Intros 02:35 Live in the Future 03:58 Will AI Outsmart us? 07:43 In the Anthropic Breadline 09:59 The Tech Genie Is Out 12:33 We Invested in COVID?! 14:25 Good Writing Is Novelty 18:50 Living Like It’s 2028 24:32 Truth dot ai 30:18 Does China have the Weights? 35:38 Everyone has AI Anxiety 39:32 Have Your Agent Talk to My Agent 42:01 What if Open Source takes the Lead? 44:03 The Sun is Setting on Google 48:00 Ride the AGI 50:46 Will There be Startups? 54:05 Defending Taiwan 1:00:05 The California Empire 1:01:26 If the U.S. Falls 1:03:11 Universal Basic Robot 1:06:01 Humans as AI Handlers
1:07:52

New podcast with Garry Tan, farbood and Daniel Francis. Live in the Future! 00:00 Guest Intros 02:35 Live in the Future 03:58 Will AI Outsmart us? 07:43 In the Anthropic Breadline 09:59 The Tech Genie Is Out 12:33 We Invested in COVID?! 14:25 Good Writing Is Novelty 18:50 Living Like It’s 2028 24:32 Truth dot ai 30:18 Does China have the Weights? 35:38 Everyone has AI Anxiety 39:32 Have Your Agent Talk to My Agent 42:01 What if Open Source takes the Lead? 44:03 The Sun is Setting on Google 48:00 Ride the AGI 50:46 Will There be Startups? 54:05 Defending Taiwan 1:00:05 The California Empire 1:01:26 If the U.S. Falls 1:03:11 Universal Basic Robot 1:06:01 Humans as AI Handlers

Naval

746,837 görüntüleme • 1 ay önce

tylercowen is bullish on AI education — here's why. 00:00 -- Preview 00:24 -- President Carlos Carvalho's AI-generated intro 03:21 -- Cowen reacts to UATX's campus 04:38 -- The AI revolution is here. Who will lose the most? 06:05 -- AI lawyers 07:17 -- Don't underestimate this 10:41 -- Changes to the "upper upper middle class" 12:38 -- How to be successful 13:43 -- The rise of managerial empires 14:02 -- When will we have the first billion dollar company with one employee? 16:05 -- 10-20 year forecast 16:19 -- Why education is so behind 17:01 -- Should you be bullish on UATX? 18:36 -- Should you still read Homer? 21:50 -- Write to think 25:01 -- Meet more people 25:42 -- How to get hired 26:54 -- Is AI your best mentor? 38:17 -- How to curb cheating 39:02 -- The new life of the mind 42:34 -- Q&A: Will there be more status associated with real education or AI education? 45:50 -- Q&A: Why do tech-savvy students need to practice using AI? 47:56 -- Q&A: Do LLMs atrophy your mind? 49:29 -- Q&A: How do you avoid AI-dependency? 51:05 -- Q&A: Isn't this vision lonely and isolating? 53:06 -- Q&A: Do students need teachers? 55:36 -- Q&A: What are the four most important courses for undergrads? 57:49 -- Q&A: Which AI company will win the AI race in the next five years and why? 59:22 -- Q&A: Can AI teach religion? 01:01:32 -- Q&A: Will AI narrow or widen our world? 01:04:37 -- Q&A: What makes us human? 01:05:42 -- Q&A: What is art? 01:08:33 -- Q&A: It's easy to catch cheaters
1:10:31

tylercowen is bullish on AI education — here's why. 00:00 -- Preview 00:24 -- President Carlos Carvalho's AI-generated intro 03:21 -- Cowen reacts to UATX's campus 04:38 -- The AI revolution is here. Who will lose the most? 06:05 -- AI lawyers 07:17 -- Don't underestimate this 10:41 -- Changes to the "upper upper middle class" 12:38 -- How to be successful 13:43 -- The rise of managerial empires 14:02 -- When will we have the first billion dollar company with one employee? 16:05 -- 10-20 year forecast 16:19 -- Why education is so behind 17:01 -- Should you be bullish on UATX? 18:36 -- Should you still read Homer? 21:50 -- Write to think 25:01 -- Meet more people 25:42 -- How to get hired 26:54 -- Is AI your best mentor? 38:17 -- How to curb cheating 39:02 -- The new life of the mind 42:34 -- Q&A: Will there be more status associated with real education or AI education? 45:50 -- Q&A: Why do tech-savvy students need to practice using AI? 47:56 -- Q&A: Do LLMs atrophy your mind? 49:29 -- Q&A: How do you avoid AI-dependency? 51:05 -- Q&A: Isn't this vision lonely and isolating? 53:06 -- Q&A: Do students need teachers? 55:36 -- Q&A: What are the four most important courses for undergrads? 57:49 -- Q&A: Which AI company will win the AI race in the next five years and why? 59:22 -- Q&A: Can AI teach religion? 01:01:32 -- Q&A: Will AI narrow or widen our world? 01:04:37 -- Q&A: What makes us human? 01:05:42 -- Q&A: What is art? 01:08:33 -- Q&A: It's easy to catch cheaters

University of Austin (UATX)

27,770 görüntüleme • 6 ay önce

It's brain versus AI time. Here's our full episode with Sebastian Seung. He's started a new company to reverse engineer the brain and master the intelligence created by nature with the hopes of informing the intelligence we've created with machines. Sebastian has done pioneering AI and neuroscience work and is now blending the two to make a digital brain. Mostly, he's fun to listen to. The Core Memory podcast is made by possible by the intelligent finance powerhouse Brex and the people who back the biggest ideas @e1ventures. Timestamps 00:00:00 - Intro 00:01:42 - Who is Sebastian Seung? 00:03:23 - What is a Connectome? 00:07:08 - The Connectome Winter 00:09:08 - Diamond Knives & AI 00:12:10 - When Will We Map the Human Brain? 00:18:09 - The "Brain Crystal" & Visual Neurons 00:26:31 - New Startup 00:33:55 - Simulating a Mind in Software 00:37:40 - Biology vs. Nvidia 00:44:04 - Is AI a Religious Quest? 00:50:05 - Mind Uploading & Transhumanism 00:59:26 - Brain-Computer Interfaces 01:06:41 - From Physics to Neuroscience 01:12:22 - Will Uploads Be Conscious? 01:19:37 - Neuromorphic Chips & Future Robotics
1:28:39

It's brain versus AI time. Here's our full episode with Sebastian Seung. He's started a new company to reverse engineer the brain and master the intelligence created by nature with the hopes of informing the intelligence we've created with machines. Sebastian has done pioneering AI and neuroscience work and is now blending the two to make a digital brain. Mostly, he's fun to listen to. The Core Memory podcast is made by possible by the intelligent finance powerhouse Brex and the people who back the biggest ideas @e1ventures. Timestamps 00:00:00 - Intro 00:01:42 - Who is Sebastian Seung? 00:03:23 - What is a Connectome? 00:07:08 - The Connectome Winter 00:09:08 - Diamond Knives & AI 00:12:10 - When Will We Map the Human Brain? 00:18:09 - The "Brain Crystal" & Visual Neurons 00:26:31 - New Startup 00:33:55 - Simulating a Mind in Software 00:37:40 - Biology vs. Nvidia 00:44:04 - Is AI a Religious Quest? 00:50:05 - Mind Uploading & Transhumanism 00:59:26 - Brain-Computer Interfaces 01:06:41 - From Physics to Neuroscience 01:12:22 - Will Uploads Be Conscious? 01:19:37 - Neuromorphic Chips & Future Robotics

Ashlee Vance

38,839 görüntüleme • 7 ay önce

I recently sat down with João Filipe and revealed my exact plan to build a $100M AI company. This is more than I've ever revealed publicly before. 0:00 Trailer 2:34 Credentials 5:56 Running multiple companies at once 11:45 Why I love content 13:33 My background 20:11 Don't quit your job *yet* 25:24 Opportunity cost 31:39 Self-reflection 36:07 Owning your AI memory 39:02 Using AI for content 52:22 Audit your life 55:39 The one AI system everyone should build 58:07 Traits > Skills 58:47 Founding AI Edge 1:01:00 How to spot trends early 1:14:00 Reinvesting in AI Edge 1:20:01 Spending $100k +/mo on AI 1:32:02 𝕏 vs. YouTube 1:49:32 My take on "AI influencers" 1:50:29 Biggest takeaway from the podcast 1:53:49 Health is the best investment
1:55:58

I recently sat down with João Filipe and revealed my exact plan to build a $100M AI company. This is more than I've ever revealed publicly before. 0:00 Trailer 2:34 Credentials 5:56 Running multiple companies at once 11:45 Why I love content 13:33 My background 20:11 Don't quit your job *yet* 25:24 Opportunity cost 31:39 Self-reflection 36:07 Owning your AI memory 39:02 Using AI for content 52:22 Audit your life 55:39 The one AI system everyone should build 58:07 Traits > Skills 58:47 Founding AI Edge 1:01:00 How to spot trends early 1:14:00 Reinvesting in AI Edge 1:20:01 Spending $100k +/mo on AI 1:32:02 𝕏 vs. YouTube 1:49:32 My take on "AI influencers" 1:50:29 Biggest takeaway from the podcast 1:53:49 Health is the best investment

Miles Deutscher

57,336 görüntüleme • 1 ay önce

Another mindblowing conversation with my good friend .Emad... Enjoy!! 00:00 - Introduction 00:22 - AI: The Biggest Shift in Human History 00:42 - AI’s Impact on Society and the Economy 01:03 - Conversation with Emad Mostaque Begins 01:50 - The Acceleration of AI and Economic Takeoff 03:07 - AI Intelligence: Beyond Human IQ 04:08 - The Rise of AI Chefs and Super Cooks 05:07 - Breaking AI Constraints: Compute and Energy 07:03 - The Future of AI: Ubiquitous Intelligence 08:04 - The Shift to Local AI Models 10:07 - Why Has Apple Lagged in AI? 11:21 - The AI Race: OpenAI, Grok, Gemini, and More 13:11 - China’s Open-Source AI Strategy 14:57 - AI Bias and Ethical Challenges 16:57 - AI’s Cross-Pollination and Memory 18:02 - Are AI Models Becoming Self-Aware? 19:16 - AI, Bitcoin, and Self-Sustaining Algorithms 21:26 - AI-Driven Economies and Autonomous Companies 23:41 - The Future of Labor: A World Without Jobs? 25:26 - AI-Powered Robots: The Next Workforce Revolution 27:28 - The End of Traditional Economic Models 30:27 - The Political Shift: Humanist vs. Transhumanist 33:04 - AI in Financial Markets: The End of Human Traders? 36:03 - The Evolution of Investing in an AI World 38:33 - AI’s Impact on Capital Formation and Business Disruption 40:01 - The Rise of Digital Twins and Post-Capital Society 42:45 - Building AI for Education, Healthcare, and Governance 46:42 - The Future of Money in an AI-Driven World 50:11 - Universal Basic AI: A New Economic Model 54:29 - The Deflationary Impact of AI and Crypto’s Role 57:02 - The AI Singularity: Five Years Until Everything Changes 58:56 - The Road Ahead: AI, Crypto, and the Future of Civilization 01:02:24 - Final Thoughts: The Most Exciting and Terrifying Time in History
1:04:51

Another mindblowing conversation with my good friend .Emad... Enjoy!! 00:00 - Introduction 00:22 - AI: The Biggest Shift in Human History 00:42 - AI’s Impact on Society and the Economy 01:03 - Conversation with Emad Mostaque Begins 01:50 - The Acceleration of AI and Economic Takeoff 03:07 - AI Intelligence: Beyond Human IQ 04:08 - The Rise of AI Chefs and Super Cooks 05:07 - Breaking AI Constraints: Compute and Energy 07:03 - The Future of AI: Ubiquitous Intelligence 08:04 - The Shift to Local AI Models 10:07 - Why Has Apple Lagged in AI? 11:21 - The AI Race: OpenAI, Grok, Gemini, and More 13:11 - China’s Open-Source AI Strategy 14:57 - AI Bias and Ethical Challenges 16:57 - AI’s Cross-Pollination and Memory 18:02 - Are AI Models Becoming Self-Aware? 19:16 - AI, Bitcoin, and Self-Sustaining Algorithms 21:26 - AI-Driven Economies and Autonomous Companies 23:41 - The Future of Labor: A World Without Jobs? 25:26 - AI-Powered Robots: The Next Workforce Revolution 27:28 - The End of Traditional Economic Models 30:27 - The Political Shift: Humanist vs. Transhumanist 33:04 - AI in Financial Markets: The End of Human Traders? 36:03 - The Evolution of Investing in an AI World 38:33 - AI’s Impact on Capital Formation and Business Disruption 40:01 - The Rise of Digital Twins and Post-Capital Society 42:45 - Building AI for Education, Healthcare, and Governance 46:42 - The Future of Money in an AI-Driven World 50:11 - Universal Basic AI: A New Economic Model 54:29 - The Deflationary Impact of AI and Crypto’s Role 57:02 - The AI Singularity: Five Years Until Everything Changes 58:56 - The Road Ahead: AI, Crypto, and the Future of Civilization 01:02:24 - Final Thoughts: The Most Exciting and Terrifying Time in History

Raoul Pal

326,827 görüntüleme • 1 yıl önce

AGI is coming. Reid Hoffman (Reid Hoffman) just wrote the book on how to prepare. According to Reid, every major tech breakthrough (the written word, the printing press, the telephone) triggered mass fear. But, contrary to our worries, new technology tends to enhance human agency—even more so, if you know how to use it well. Reid is the cofounder of LinkedIn, Inflection AI, and Manas, a partner at Greylock Partners, an award-winning podcaster, and an early backer and board member of OpenAI. We spent an hour talking about how to develop a compass for navigating AGI. Here are a few takeaways: - Our sense of human agency is not just about external control but an internal stance—how we approach uncertainty & new tech is crucial - In new technology waves, NO blueprint or plan will have the right answers. Instead, adapting to new technology requires broad access, an experimental mindset, and flexibility - In an AGI world most jobs will transform, not disappear—and how you can prepare with hands-on trial and error - How certain social norms and ethics should change as AGI changes the landscape—like individual access to personal data - Why now may be finally be the era where quantified self tools become valuable …and more, including everything in his new book Superagency, out this week. It was a pleasure to have him on the show for a second time. This is a must-watch for anyone who wants to help build a more human future with AI. Watch below! Timestamps: Introduction: 00:01:29 Patterns in how we’ve historically adopted technology: 00:02:50 Why humans have typically been fearful of new technologies: 00:07:02 How Reid developed his own sense of agency: 00:13:25 The way Reid thinks about making investment decisions: 00:20:08 AI as a “techno-humanist” compass: 00:29:40 How to prepare yourself for the way AI will change knowledge work: 00:35:30 Why equitable access to AI is important: 00:41:39 Reid’s take on why private commons will be beneficial for society: 00:45:15 How AI is making Silicon Valley’s conception of the “quantified self” a reality: 00:47:23 The shift from symbolic to sub-symbolic AI mirrors how we understand intelligence: 00:52:14 Reid’s new book, Superagency: 01:03:29
1:09:23

AGI is coming. Reid Hoffman (Reid Hoffman) just wrote the book on how to prepare. According to Reid, every major tech breakthrough (the written word, the printing press, the telephone) triggered mass fear. But, contrary to our worries, new technology tends to enhance human agency—even more so, if you know how to use it well. Reid is the cofounder of LinkedIn, Inflection AI, and Manas, a partner at Greylock Partners, an award-winning podcaster, and an early backer and board member of OpenAI. We spent an hour talking about how to develop a compass for navigating AGI. Here are a few takeaways: - Our sense of human agency is not just about external control but an internal stance—how we approach uncertainty & new tech is crucial - In new technology waves, NO blueprint or plan will have the right answers. Instead, adapting to new technology requires broad access, an experimental mindset, and flexibility - In an AGI world most jobs will transform, not disappear—and how you can prepare with hands-on trial and error - How certain social norms and ethics should change as AGI changes the landscape—like individual access to personal data - Why now may be finally be the era where quantified self tools become valuable …and more, including everything in his new book Superagency, out this week. It was a pleasure to have him on the show for a second time. This is a must-watch for anyone who wants to help build a more human future with AI. Watch below! Timestamps: Introduction: 00:01:29 Patterns in how we’ve historically adopted technology: 00:02:50 Why humans have typically been fearful of new technologies: 00:07:02 How Reid developed his own sense of agency: 00:13:25 The way Reid thinks about making investment decisions: 00:20:08 AI as a “techno-humanist” compass: 00:29:40 How to prepare yourself for the way AI will change knowledge work: 00:35:30 Why equitable access to AI is important: 00:41:39 Reid’s take on why private commons will be beneficial for society: 00:45:15 How AI is making Silicon Valley’s conception of the “quantified self” a reality: 00:47:23 The shift from symbolic to sub-symbolic AI mirrors how we understand intelligence: 00:52:14 Reid’s new book, Superagency: 01:03:29

Dan Shipper 📧

47,209 görüntüleme • 1 yıl önce

New episode with Dr. Konrad Kording (Kording Lab 🦖), professor of bioengineering and neuroscience at the University of Pennsylvania (Penn) and co-director of CIFAR's Learning in Machines & Brains program (CIFAR). Konrad works at the intersection of causality, machine learning, and neuroscience, building rigorous methods for causal reasoning when experiments aren't possible — and challenging how researchers interpret neural data and build AI. Konrad argues the most promising path to understanding how the brain works is to read the brain’s wiring directly, down to the molecular detail of each connection, and to build compilers and simulations to understand the brain’s computation directly. In this episode we go deep into how neurons work, how neurons wire together, and how organic and artificial neural networks differ. We discuss why organic neurons are doing much more; how a model of a single organic neuron can solve MNIST — computing more like a 3-layer artificial neural network; how the brain might learn by solving credit assignment with only local signals; how to approximate backprop without a global algorithm; why AI and humans are intelligent along different dimensions; why Konrad isn’t very worried about AI replacing us; economic models of intelligence and physical work; and much more. Konrad is a brilliant, contrarian thinker who explains complex concepts very intuitively. It is a solid computational neuroscience primer. I hope you enjoy this conversation as much as I did! Other links to this episode and references below. Chapters 00:00:00 Introduction 00:01:01 How organic neurons work 00:24:13 How the brain learns: circuits and credit assignment 00:45:29 Recording the brain 00:52:47 Why simulating brains is hard 01:05:00 A new approach: connectomes and compilers 01:21:00 Why simulate brains? 01:29:50 How AI and human intelligence differ 01:41:04 Evolution, intelligence and AI risk 01:52:42 Robotics, causality, and the roots of intelligence 02:05:53 AI for science and scientific rigor 02:13:05 The economics of intelligence 02:27:50 A hopeful future
2:29:40

New episode with Dr. Konrad Kording (Kording Lab 🦖), professor of bioengineering and neuroscience at the University of Pennsylvania (Penn) and co-director of CIFAR's Learning in Machines & Brains program (CIFAR). Konrad works at the intersection of causality, machine learning, and neuroscience, building rigorous methods for causal reasoning when experiments aren't possible — and challenging how researchers interpret neural data and build AI. Konrad argues the most promising path to understanding how the brain works is to read the brain’s wiring directly, down to the molecular detail of each connection, and to build compilers and simulations to understand the brain’s computation directly. In this episode we go deep into how neurons work, how neurons wire together, and how organic and artificial neural networks differ. We discuss why organic neurons are doing much more; how a model of a single organic neuron can solve MNIST — computing more like a 3-layer artificial neural network; how the brain might learn by solving credit assignment with only local signals; how to approximate backprop without a global algorithm; why AI and humans are intelligent along different dimensions; why Konrad isn’t very worried about AI replacing us; economic models of intelligence and physical work; and much more. Konrad is a brilliant, contrarian thinker who explains complex concepts very intuitively. It is a solid computational neuroscience primer. I hope you enjoy this conversation as much as I did! Other links to this episode and references below. Chapters 00:00:00 Introduction 00:01:01 How organic neurons work 00:24:13 How the brain learns: circuits and credit assignment 00:45:29 Recording the brain 00:52:47 Why simulating brains is hard 01:05:00 A new approach: connectomes and compilers 01:21:00 Why simulate brains? 01:29:50 How AI and human intelligence differ 01:41:04 Evolution, intelligence and AI risk 01:52:42 Robotics, causality, and the roots of intelligence 02:05:53 AI for science and scientific rigor 02:13:05 The economics of intelligence 02:27:50 A hopeful future

Juan Benet

49,297 görüntüleme • 1 ay önce

.Dean W. Ball is one of the most famous opponents of AI regulation, and co-author of America's AI strategy. But unlike many new AI commentators he's a true intellectual and a blogger at heart — not a shallow ideologue or corporate mouthpiece. So he doesn't wave away concerns and predict a smooth simple ride. In fact, for Dean an unpredictable or volatile future is the reason to take as few big steps now as possible. He fears the wrong regulations, deployed too early, would "in a Shakespearean fashion, bring about the world that we do not want." More specifically, premature regulation might lock us into addressing the wrong problem (e.g. rogue AI when the real issue is power concentration), with the wrong target (e.g. models rather than companies), through the wrong institutions (e.g. AI-specific bodies that are captured by industry), while making it harder to build the actual solutions we'll need (e.g. open source or legal mechanisms newly enabled by AI). I booked an interview with Dean so I could thoroughly read his Substack, get to grips with his worldview, and figure out why I often see things differently. We cover: We'll get superintelligence, but it will probably be steerable (00:01:54) AI won't be militarised fast (00:11:10) AI self-improvement matters but is no game changer (00:28:58) The case for regulating at the last possible moment (00:33:51) AI could destroy our fragile democratic equilibria. So why not freak out? (00:53:41) Why Dean fears AI will soon be way overregulated (01:04:10) How to handle the real risks with minimal collateral damage (01:16:23) Easy wins against AI misuse (01:28:27) A company would be sued for trillions if their AI caused a pandemic (01:49:43) Dean dislikes compute thresholds and would do this instead. (01:59:06) Dean expects a MAGA-Yudkowskyite alliance. But Doomers and E/accs are more alike than different. (02:14:40) A tactical case for focusing on present-day harms (02:29:16) Is there any way to get the US government to actually use AI in its work? (02:47:43) On the 80,000 Hours Podcast. Links below — enjoy!
2:57:11

.Dean W. Ball is one of the most famous opponents of AI regulation, and co-author of America's AI strategy. But unlike many new AI commentators he's a true intellectual and a blogger at heart — not a shallow ideologue or corporate mouthpiece. So he doesn't wave away concerns and predict a smooth simple ride. In fact, for Dean an unpredictable or volatile future is the reason to take as few big steps now as possible. He fears the wrong regulations, deployed too early, would "in a Shakespearean fashion, bring about the world that we do not want." More specifically, premature regulation might lock us into addressing the wrong problem (e.g. rogue AI when the real issue is power concentration), with the wrong target (e.g. models rather than companies), through the wrong institutions (e.g. AI-specific bodies that are captured by industry), while making it harder to build the actual solutions we'll need (e.g. open source or legal mechanisms newly enabled by AI). I booked an interview with Dean so I could thoroughly read his Substack, get to grips with his worldview, and figure out why I often see things differently. We cover: We'll get superintelligence, but it will probably be steerable (00:01:54) AI won't be militarised fast (00:11:10) AI self-improvement matters but is no game changer (00:28:58) The case for regulating at the last possible moment (00:33:51) AI could destroy our fragile democratic equilibria. So why not freak out? (00:53:41) Why Dean fears AI will soon be way overregulated (01:04:10) How to handle the real risks with minimal collateral damage (01:16:23) Easy wins against AI misuse (01:28:27) A company would be sued for trillions if their AI caused a pandemic (01:49:43) Dean dislikes compute thresholds and would do this instead. (01:59:06) Dean expects a MAGA-Yudkowskyite alliance. But Doomers and E/accs are more alike than different. (02:14:40) A tactical case for focusing on present-day harms (02:29:16) Is there any way to get the US government to actually use AI in its work? (02:47:43) On the 80,000 Hours Podcast. Links below — enjoy!

Rob Wiblin

28,859 görüntüleme • 8 ay önce

Ryan Greenblatt is lead author of "Alignment faking in LLMs" and one of AI's most productive researchers. He puts a 25% probability on automating AI research by 2029. We discuss: • Concrete evidence for and against AGI coming soon • The 4 easiest ways for AI to take over • What evidence we have on how fast / long the intelligence explosion will go • Would misaligned AGI go rogue early or bide its time • Whether 'pause at human level' is naive or smart • Lots more. My head was often spinning during this interview, in a good way. Find it on the 80,000 Hours Podcast, links below. Enjoy! 1:29 How close are we to automating AI R&D? 5:15 Really, though: how capable are today's models? 13:01 Why AI companies get automated first 18:10 Most likely ways for AGI to take over 30:04 Would AGI go rogue early or bide its time? 34:53 "Pause at human level" 46:43 AI control vs AI alignment 52:38 Do we have to hope to catch AIs red-handed? 56:57 How would a slow AGI takeoff look? 1:05:04 Why might an intelligence explosion not happen for 8+ years? 1:17:05 Key challenges in forecasting AI progress 1:25:07 The bear case on AGI 1:30:59 The change to "compute at inference" 1:36:38 How much has pretraining petered out? 1:49:08 Could we get an intelligence explosion within a year? 1:53:08 Reasons AIs might struggle to replace humans 2:00:10 Things could go insanely fast when we automate AI R&D. Or not. 2:14:52 How fast would the intelligence explosion slow down? 2:27:53 Bottom line for mortals 2:34:00 Six orders of magnitude of progress... what does that even look like? 2:44:10 Neglected and important technical work people should be doing 2:48:16 What's the most promising work in governance? 2:51:37 Ryan's current research priorities
2:54:26

Ryan Greenblatt is lead author of "Alignment faking in LLMs" and one of AI's most productive researchers. He puts a 25% probability on automating AI research by 2029. We discuss: • Concrete evidence for and against AGI coming soon • The 4 easiest ways for AI to take over • What evidence we have on how fast / long the intelligence explosion will go • Would misaligned AGI go rogue early or bide its time • Whether 'pause at human level' is naive or smart • Lots more. My head was often spinning during this interview, in a good way. Find it on the 80,000 Hours Podcast, links below. Enjoy! 1:29 How close are we to automating AI R&D? 5:15 Really, though: how capable are today's models? 13:01 Why AI companies get automated first 18:10 Most likely ways for AGI to take over 30:04 Would AGI go rogue early or bide its time? 34:53 "Pause at human level" 46:43 AI control vs AI alignment 52:38 Do we have to hope to catch AIs red-handed? 56:57 How would a slow AGI takeoff look? 1:05:04 Why might an intelligence explosion not happen for 8+ years? 1:17:05 Key challenges in forecasting AI progress 1:25:07 The bear case on AGI 1:30:59 The change to "compute at inference" 1:36:38 How much has pretraining petered out? 1:49:08 Could we get an intelligence explosion within a year? 1:53:08 Reasons AIs might struggle to replace humans 2:00:10 Things could go insanely fast when we automate AI R&D. Or not. 2:14:52 How fast would the intelligence explosion slow down? 2:27:53 Bottom line for mortals 2:34:00 Six orders of magnitude of progress... what does that even look like? 2:44:10 Neglected and important technical work people should be doing 2:48:16 What's the most promising work in governance? 2:51:37 Ryan's current research priorities

Rob Wiblin

34,618 görüntüleme • 1 yıl önce