Rob Wiblin
@robertwiblin • 51,139 subscribers
Host of the 80,000 Hours Podcast. Exploring the inviolate sphere of ideas one interview at a time: https://t.co/2YMw00bkIQ
Videos
One OpenAI critic calls it "the theft of at least the millennium and quite possibly all of human history." Are they right? UCLA nonprofit law expert Rose Chan Loui and I discuss OpenAI's unprecedented gambit to ditch its nonprofit for good: 4:08 – How OpenAI carefully chose a complex nonprofit structure that failed 11:56 – OpenAI's new plan to go for-profit 14:49 – The nonprofit board is out-gunned, out-manned, out-numbered, out-planned. Who can help? 17:28 – Who would stand cheated in a bad for-profit switch? 28:03 – Has this happened before? 29:37 – Is it truly in the nonprofit's interest to sell control of OpenAI, or is it a bad deal? 36:10 – The difficulty of valuing OpenAI's future windfall profits 42:31 – Control of OpenAI is independently incredibly valuable and demands compensation 53:09 – What most miss: It's very important the nonprofit get cash and not just equity 1:05:36 – Is it a farce to call this an "arm's-length transaction"? 1:10:59 – How the nonprofit board can best play their hand 1:17:42 – Can Elon Musk mount a court challenge and how that would work The volunteer nonprofit board is hugely outgunned by groups who would profit by tens of billions by screwing it over. And it will take a heroic effort and help from some state attorneys general to get everything it's owed. But that... just might happen. We don't hold back on the opinions. Episode available in audio and video on the 80,000 Hours Podcast here, on any app, or YouTube.
Rob Wiblin7,047,135 Aufrufe • vor 1 Jahr
I see 7 actual updates relevant to AGI timelines this year: 1. Exploding revenue and GPU rental $ 2. METR time horizons saturate 3. The Mythos 'jump' 4. Sparks of RSI 5. AI still sucks at business 6. AI makes some math breakthroughs 7. Inference scaling remains cheap I wanted to understand whether these things truly show what people think they show, and how they should affect my timelines, so I made this video. On balance my timelines have contracted about 1 year since last December. IMO these are the 4 topics that most split AGI bulls from bears: 1. What skills are needed for RSI 2. Whether AI still lacks crucial skills in areas with poor feedback density 3. Whether RLVR will generalise to messy domains 4. Whether RSI will produce a positive feedback loop It's no surprise people disagree about this stuff as there's little public evidence to settle them either way. On the 80,000 Hours Podcast, links below. Enjoy!
Rob Wiblin48,468 Aufrufe • vor 16 Tagen
The 14 most common ways we're all screwing up AGI forecasting according to Toby Ord (my wording based on our interview): 1. Believing AI research is just hill-climbing 2. Imagining AI research is mostly programming 3. Forecasting 'could' instead of 'will' 4. Thinking the current benchmarks are the last ones 5. Extrapolating trends out to a finish line when the finish line is unknown 6. Assuming inputs scaling at the same rate forever 7. Conflating intelligence and capability 8. Consuming point estimates and discarding the error bars 9. Dismissing dissenting experts 10. Forecasting very different things using the same words 11. Assuming different capabilities arrive simultaneously 2. Treating 'we don’t know' as permission to carry on as usual 13. Choosing a plan of action that minimises regret rather than maximises impact 14. Taking surface model impressiveness at face value In our convo Toby Ord also makes the case that: • AI self-improvement is uniquely dangerous in 4 ways, but also might not even work • A ban on superintelligence is possible • A US-China treaty on superintelligence is also possible • ‘Broad timelines’ are what we should act on • Transformative AI is likely a decade away • We should just ban unmonitorable chain-of-thought today On the 80,000 Hours Podcast wherever you get podcasts, links below. Enjoy!
Rob Wiblin23,451 Aufrufe • vor 9 Tagen
My best interview in some time. Rohin Shah leads AGI alignment/safety at DeepMind. And he has a lot of spicy personal takes: We probably won’t get catastrophic misalignment (00:49) Safety 'commitments' have severe limitations (10:38) The intelligence explosion probably isn't imminent (1:52:44) Why he's not working to pause AI advances (51:44) Pre-deployment evals aren't the right focus (for catastrophic risks) (37:41) Signalling concern for safety sometimes diverts resources from actually making AI safe (01:09:51) Reading AI thoughts is v useful for safety – and we'll probably be able to for years to come (54:17) Governance is somewhat more likely to be the bottleneck than alignment (43:55) Rohin's team doesn't have a veto, and that's OK (27:36) Central banks are a promising model for regulating AI (33:34) Also: Google DeepMind's actual plan for building AGI safely (1:40:29) How external researchers can positively influence big AI companies (2:21:55) The roles GDM most needs to hire for (2:37:03) On the 80,000 Hours Podcast. Links below - enjoy! (Rohin Shah)
Rob Wiblin156,664 Aufrufe • vor 2 Monaten
Every AI lab is working to make their AI helpful, harmless and honest. Max Harms (Max Harms) thinks this is a complete wrong turn, and 'aligning' AI to human values is actively dangerous. In his view a safe AGI must have absolutely no opinion about how the world ought to be, be willingly modifiable, and be entirely indifferent to being shut down. The opposite of all commercial models today. The key appeal is that so-called 'corrigibility' could be an attractor state – get close enough and the AI actively helps you make it more corrigible over time. That forgiveness would at least give us a shot. It's a strategy that feels natural within the 'MIRI worldview', recently laid out by his colleagues Eliezer Yudkowsky ⏹️ and Nate Soares ⏹️ in 'If Anyone Builds It Everyone Dies'. But it risks causing a different AI catastrophe, because the resulting AI model would necessarily be willing to assist any human operator with a power grab, or indeed any crime at all. I interviewed Max on the 80,000 Hours Podcast to debate the MIRI worldview, and what we should do to figure out if corrigibility ought to be our one and only focus. Links below – enjoy! 00:01:56 If anyone builds it, will everyone die? The MIRI perspective on AGI risk 00:24:28 Evolution failed to ‘align’ us, just as we'll fail to align AI 00:42:56 We're training AIs to want to stay alive and value power for its own sake 00:52:24 Objections: Is the 'squiggle/paperclip problem' really real? 01:05:02 Can we get empirical evidence re: 'alignment by default'? 01:10:17 Why do few AI researchers share Max's perspective? 01:18:34 We're training AI to pursue goals relentlessly — and superintelligence will too 01:24:51 The case for a radical slowdown 01:27:53 Max's best hope: corrigibility as stepping stone to alignment 01:32:34 Corrigibility is both uniquely valuable, and practical, to train 01:45:06 What training could ever make models corrigible enough? 01:51:38 Corrigibility is also terribly risky due to misuse risk 01:58:57 A single researcher could make a corrigibility benchmark. Nobody has. 02:12:20 Red Heart & why Max writes hard science fiction 02:34:08 Should you homeschool? Depends how weird your kids are.
Rob Wiblin296,652 Aufrufe • vor 5 Monaten
'MIT Study Shows 95% of AI Projects Lose Money' was the #1 AI meme for the public (and politicians) last year. So I looked into this 'study'. It was… much worse than I would have guessed. And I suspect not by mistake. The authors had a hidden agenda from the start. I explain:
Rob Wiblin112,320 Aufrufe • vor 3 Monaten
This tweet got over 1M views so we made it a video: How much money does Meta make by enabling crimes? "Internal docs leaked to Reuters show: • 10% of all Meta revenue comes from ads for scams & banned goods ($16B/year) • Meta estimates it's involved in 1/3 of all successful scams in the US • That suggests they drive $50B in scam losses for US consumers alone each year • Meta earns ~$3B annually from scam/banned goods ads run by Chinese operations alone..."
Rob Wiblin115,517 Aufrufe • vor 5 Monaten
2:33:25
Sensitive content
This media may contain sensitive content.
Convention wisdom is that bioweapons are humanity's greatest weakness – 100x cheaper to make than to defend against. Andrew Snyder-Beattie thinks conventional wisdom is likely wrong. He has a plan cheap enough to do without government. Useful even in worst case scenarios like mirror bacteria. Effective enough to save most people. In one of my all-time fav interviews he lays out a low-tech 4-step approach developed by his research team at Open Philanthropy, to fix a problem most have thought unsolvable. ASB is hiring for many roles in this project from logistics to biotech to manufacturing, and has $100s millions to deploy. Enjoy, links below! 2:10 How bad it could get 9:19 The worst-case scenario: mirror bacteria 18:14 Why low-tech 25:30 Prevention 31:21 The “4 pillars” plan 33:09 ASB is hiring now to make this happen 35:11 Everyone was wrong: biorisks are defence dominant 40:23 Pillar 1: Lungs 55:53 Pillar 2: Biohardening 1:15:19 Pillar 3: Detection 1:28:40 Pillar 4: The wrench hypothesis 1:40:12 The plan's biggest weaknesses 1:44:44 Would chaos make this impossible to pull off? 1:51:50 Would rogue AI make bioweapons? 1:57:57 We can feed the world even if all the plants die 2:07:03 Could a bioweapon make the Earth uninhabitable? 2:09:35 What ASB is hiring for 2:30:27 How to protect yourself and your family (On the 80,000 Hours Podcast, available anywhere you get podcasts.)
Rob Wiblin168,670 Aufrufe • vor 10 Monaten
Yoshua Bengio thinks he knows how to make provably safe superintelligent agents. Bengio built the foundations of modern AI and is the most cited living scientist. He believes his alternative training setup would: 1. Guarantee honesty 2. Prevent unintended goals 3. Produce capable agents 4. Port over most data and techniques from current LLMs 5. Not be inherently more expensive, and perhaps be more intelligent Bengio claims the honesty and lack of unintended goals can be proven mathematically, at least given particular assumptions. And his new organization, LawZero, is aiming to build a scrappy prototype as soon as possible. The architecture is called 'Scientist AI' and it's based on training a model to explain empirical observations, including what people say, rather than training AIs that mimic human behaviour or seek our approval. (Bengio's frank assessment is that "reinforcement learning is evil" and that allowing AIs to independently train their successors is "the most crazy, dangerous bet that unfortunately we are on track to do.") But skeptics question whether Scientist AI really does solve the fundamental problem of 'eliciting latent knowledge' from AI models. And with the commercial race for superintelligence so intense, it's not clear whether the proposal will be able to compete or have time to bear fruit, even if it's sound in theory. On The 80,000 Hours Podcast, links below – enjoy! • Making AI honest and safe (00:00:00) • Scientist AI in plain English (00:02:27) • How Scientist AI differs from LLMs (00:06:32) • How the training data works (00:14:02) • Can this become an agent? (00:21:02) • Why Yoshua is now more optimistic (00:32:11) • Why companies can’t stop racing (00:36:35) • A working prototype won't take long (00:49:15) • Scientist models might be more capable (00:53:34) • “Reinforcement learning is evil” (01:01:27) • Scientist AI from guardrail to agent (01:08:37) • Can safe AI still be competent? (01:12:38) • How much will this cost? (01:19:29) • Can it generalise beyond maths and science? (01:23:26) • A multi-national push for superintelligence (01:39:19) • Want to work with or fund Yoshua? (01:51:16) • Why smart people ignore AI risk (01:54:45) • Don’t let AI build the next AI (02:01:33) • Why politicians miss the real risks (02:12:28) • Why Yoshua changed his mind about AI risk (02:21:27)
Rob Wiblin65,088 Aufrufe • vor 3 Monaten
I spent the last 2 days trying to figure out how scary Claude Mythos is. I think it's fairly scary, though not because of the hacking: 1. It indicates fully-automated AI R&D is coming sooner 2. Its alignment seems better, which is good. But all the alignment tests have serious flaws, which is bad. 3. There are a few specific warning signs Mythos might not be trustworthy I explain what stood out to me in the 244-page System Card and 59-page Alignment Risk Report in this essay for the 80,000 Hours Podcast. Judging by those 2 reports Anthropic itself seems kinda scared of Claude now. And I'm sure views vary widely within the company, but at times it feels like they only give themselves a 50/50 chance of being able to keep the next few Claudes fully under control. So I guess we're on the same page! If it does turn out we can take the safety results at face value we may look back and see this week as watershed good news. If they can't, the opposite. (I wish I had had more time to look into how reassuring it is that their automated monitoring systems don't seem to be picking up much misbehaviour post internal-deployment. I think that's what someone at Anthropic who feels more relaxed would point to. Next time!) Links below - enjoy!
Rob Wiblin60,204 Aufrufe • vor 4 Monaten
Tackling NIMBYs head on has been an abject failure. You don't smash them — sooner or later they smash you. We need a different approach. And there’s a weirdly obvious solution to fighting NIMBY: 'stuffing their mouths with gold'. Sam Bowman (Sam Bowman) and I discuss his 'housing theory of everything' and how to actually fix our catastrophically busted planning permission system on the 80,000 Hours Podcast. Also: how avant-garde architects are the great villains of our age, how to tackle terrible incumbent institutions, Europe's need for nuclear, Ozempic remaining highly underrated, and how progress studies stays sane. 1:21 We can't seem to build anything 3:15 And it's ruining people's lives 8:28 The housing theory of everything 17:03 The UK is the world's worst 35:47 Why almost no progress fixing it 43:11 NIMBYs are often harmed by development 54:50 Solution #1: Street votes 1:18:47 Will street votes come to the US 1:24:21 Solution #2: Stuffing mouths with gold 1:43:42 The most important policy setting you've never heard of 1:57:06 Solution #3: Opt-outs 2:10:32 How to make it happen 2:17:30 Making old institutions die a gradual death 2:31:11 The evil of modern architecture and the importance of beauty 2:44:14 The north needs nuclear 3:01:43 Ozempic and "the overweight theory of everything" 3:17:08 How progress studies has avoided online madness I'd been hoping to interview Sam Bowman for years and he's the guest I always dreamed he would be. Transcript and links on the 80,000 Hours site, and a much higher res 4K version on YT (can't upload full quality to Twitter sadly).
Rob Wiblin197,405 Aufrufe • vor 1 Jahr
METR investigated what a rogue AI could secretly get away with inside a frontier AI lab, in close collaboration with OpenAI, GDM, Anthropic and Meta. Including sending a red-teamer into Anthropic to playact 'evil Claude' for 3 weeks. Here's what stands out to me from their new 320-page report: 00:00 What could an unreleased AI get away with? 01:54 Motive: Why grab more compute? 05:46 Opportunity: YOLO mode and jailbreaks 11:02 Means: Brilliant idiots in data centres 15:45 We have to test unreleased models... 18:29 ...especially if AI R&D is coming in 2028
Rob Wiblin40,047 Aufrufe • vor 3 Monaten
What the hell happened with AGI timelines in 2025? Was it just vibes? Faulty analysis? Unexpected technical results? I try to make sense of what drove the wild swings in sentiment: • The great timelines contraction (00:47) • Why timelines went back out again (02:10) • Longstanding reasons AGI could take a long time (11:13) • So what's the upshot of all of these updates? (14:47) • 5 reasons the radical pessimists are still wrong (16:54) • Even long timelines are short now (23:54) (On the 80,000 Hours Podcast, links below.)
Rob Wiblin71,964 Aufrufe • vor 6 Monaten
Neel Nanda is leading a Google DeepMind research team at 26. He and I discuss: • How that happened • “If your safety work doesn't advance capabilities, it's probably bad safety work” • Should people work at the safest or most reckless AI company? • An AI PhD – with these timelines?! • How to best operate in a big frontier AI company • Neel's distinctive uses of LLMs and which cold emails he answers • A common reasoning error in AI alignment • Why he (Neel Nanda) refuses to share his p(doom) This is part 2 of our conversation, part 1 was a comprehensive update on his research area: mechanistic interpretability, which I'll link below. Links to this episode of the 80,000 Hours Podcast below — enjoy!
Rob Wiblin111,865 Aufrufe • vor 11 Monaten
The Will MacAskill episode on: 1. Why we should both pay AIs and trade with them (00:38:13) 2. Why not just push to pause AI development? (01:42:17) 3. Making AIs extremely risk averse might also make them safe (00:38:13) 4. The panic over sycophancy was justified (08:11) 5. The character we give AI models could determine the future (00:01:00) 6. The toughest trade-offs in AI personality design (13:24) 7. The effective altruism comeback (01:52:19) 8. Every population ethics recommends 'tiling the universe', except Will's new one (02:39:35) 9. Non-causal decision theory might accidentally motivate a near-best future (01:22:19) 10. How to aim at the best possible world without utopianism (02:09:30) For the 80,000 Hours Podcast, links below - enjoy! William MacAskill
Rob Wiblin40,908 Aufrufe • vor 4 Monaten
Even 'aligned AGI' naturally kills democracy and leads to oligarchy, or worse. That's the take of Anthropic's past alignment evals team lead, Prof David Duvenaud. Once humans aren't needed to do jobs or serve in the military, to governments we look like "meddlesome parasites". With voters unable to contribute but engaged in incessant activism to extract resources from others – resources the country needs to avoid domination by rivals – the attraction of mass disenfranchisement could be overwhelming. In 2025 David co-authored "Gradual Disempowerment", which aimed to lay out this and many other political, economic, and cultural forces that could sideline ordinary people (and maybe all people) in the presence of machines that can cheaply do everything humans will do. Most controversially, David and colleagues believe that competitive forces will compel disempowerment, even if all those AIs are aligned and loyal to their users. I wasn't sure how much I believed this vision of how the future might play out, so I interviewed him for The 80,000 Hours Podcast to probe how well it holds up. He and I covered: 01:30 The case that alignment isn’t enough 14:15 How smart AI advice still leads to terrible outcomes 19:05 How gradual disempowerment occurs 22:10 Economics: Humans become "meddlesome parasites" 29:37 Humans are a "criminally decadent" waste of energy 40:48 Is humans losing control actually bad, ethically? 57:47 Politics: Governments stop needing people 1:10:47 Can human culture survive in an AI-dominated world? 1:27:20 Will the future be determined by competitive or coordinative forces? 1:35:00 Can we find a single good post-AGI equilibria for humans? 1:45:17 Do we know anything useful to do about this? 1:56:42 How important is this problem compared to other AGI issues? 2:05:42 Improving global coordination may be our best bet 2:08:14 The 'Gradual Disempowerment Index' 2:11:22 The government will fight to write AI constitutions 2:17:48 “The intelligence curse” and Workshop Labs 2:23:48 Mapping out disempowerment in a world of aligned AGIs 2:30:10 What do David’s CompSci colleagues make of all this? Links below — enjoy!
Rob Wiblin58,298 Aufrufe • vor 6 Monaten
![I interviewed Holden Karnofsky of Anthropic (and past CEO of Open Philanthropy) for 4.5 hours on almost all his AGI takes: • The AI 'race' isn't a coordination failure [00:18:01] • Why he's currently focused on pursuing easy wins and partial victories [02:43:43] • "It'll be the second advanced species ever" [02:31:44] • Dozens of 'concrete shovel-ready projects' he's excited about [01:17:58] • People don't appreciate all the engaging, impactful, high-feedback work available tackling AGI risks [01:19:00] • Having a more responsible AI company really does matter [02:43:43] • You shouldn't count on trusting anyone in AI [00:44:30] • To take over an AGI might just wait it out [00:07:39] • Farm animal welfare campaigns that targeted companies contain some useful lessons here [02:34:18] • Human-AI relationships really are troubling [03:53:58] • It's totally plausible we incompetently fumble our way into a great future [03:04:16)] • AI R&D is *the* thing to worry about [01:57:31] • And many others. Holden has opinions on almost everything and isn't afraid to speak his mind. Enjoy! (On the 80,000 Hours Podcast anywhere you get podcasts. Links below.)](https://image.24vids.com/tw-1983932866540028357/amplify_video_thumb/1983903872180502528/img/eD9T3pSglJW9u5LU.jpg)
I interviewed Holden Karnofsky of Anthropic (and past CEO of Open Philanthropy) for 4.5 hours on almost all his AGI takes: • The AI 'race' isn't a coordination failure [00:18:01] • Why he's currently focused on pursuing easy wins and partial victories [02:43:43] • "It'll be the second advanced species ever" [02:31:44] • Dozens of 'concrete shovel-ready projects' he's excited about [01:17:58] • People don't appreciate all the engaging, impactful, high-feedback work available tackling AGI risks [01:19:00] • Having a more responsible AI company really does matter [02:43:43] • You shouldn't count on trusting anyone in AI [00:44:30] • To take over an AGI might just wait it out [00:07:39] • Farm animal welfare campaigns that targeted companies contain some useful lessons here [02:34:18] • Human-AI relationships really are troubling [03:53:58] • It's totally plausible we incompetently fumble our way into a great future [03:04:16)] • AI R&D is *the* thing to worry about [01:57:31] • And many others. Holden has opinions on almost everything and isn't afraid to speak his mind. Enjoy! (On the 80,000 Hours Podcast anywhere you get podcasts. Links below.)
Rob Wiblin73,723 Aufrufe • vor 9 Monaten
AI models currently have a 50% chance of doing something that takes a human expert one hour. This doubles every 7 months. In 2 years? They could automate full workdays. In 4 years? A full month. I discuss the most important graph in AI today with Beth Barnes, the CEO of METR, which uncovered this rule of AI progress. Her bottom line: "It really doesn't seem like 2 years would be surprising for recursively self-improving AI." Beth also explains: where company safety testing fails, why there are no true closed-weight models, AI undermines leading powers, why she's come around on open weighting, and why models might be about to start playing dumb much more often. Enjoy! Available on the 80,000 Hours Podcast in all apps. Links below. 1:51 Can we see AI scheming in the chain of thought? 12:50 Alignment faking 17:33 We have to test models before they're even used inside AI companies 31:56 Each 7 months models can do tasks twice as long 51:31 METR's research finds AIs are solid at AI research already 58:18 AI may turn out to be strong at novel and creative research 1:07:55 Recursively self-improving AI might even be here in two years 1:14:29 Could evaluations backfire? 1:39:55 Do we need external auditors doing AI safety tests? 1:54:09 Why not work at AI companies 2:08:40 The new more dire situation has forced changes to METR's strategy 2:21:49 Overrated: Interpretability research 2:32:55 Overrated: Major AI companies' contributions to safety research 2:39:15 Could we ban using AI to enhance AI, or is that just naive? 2:45:31 Open-weighting models is often good 2:50:22 What we can learn about AGI from the nuclear arms race 3:10:43 AI is more like bioweapons because it undermines the leading power 3:42:09 What research METR plans to do next
Rob Wiblin93,669 Aufrufe • vor 1 Jahr