Загрузка видео...

Не удалось загрузить видео

На главную

I sat down with Lukasz Kaiser to get into whether the architecture he helped invent is actually enough, and what's next in generalization, coding agents, RL and more. Lukasz co-authored "Attention Is All You Need," the paper that introduced the transformer and worked on reasoning models at OpenAI so...

jacobeffron's profile picture

Jacob Effron

4,732 subscribers

75,222 просмотров • 3 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

At OpenAI, Chief Scientist Jakub Pachocki helps lead the research roadmap to AGI including a research intern-level AI system by September 2026 and a fully automated AI researcher by March 2028. I sat down with Jakub to check on those timelines and ask him all of my top-of-mind AI questions including: ▪️ How OpenAI thinks about extending RL beyond code and math ▪️ The current state of alignment research as more powerful models loom ▪️ The future of continual learning ▪️ How startups should think about building their own models/harnesses And he also shared some great stories around OpenAI’s pioneering work on math. YouTube: Spotify: Apple: 0:00 Intro 1:53 Research Intern Capability Timelines 4:59 Math Breakthroughs 7:59 RL Beyond Verifiable Tasks 12:32 RL vs In-Context 19:01 Allocating Compute Internally 28:18 AI for Science 31:40 Pattern Matching 33:23 Solving the Hardest Math Problems 37:40 Chain of Thought Monitoring 44:33 Generalization and Value Alignment in Models 47:57 Inside OpenAI 51:55 Quickfire
58:45

At OpenAI, Chief Scientist Jakub Pachocki helps lead the research roadmap to AGI including a research intern-level AI system by September 2026 and a fully automated AI researcher by March 2028. I sat down with Jakub to check on those timelines and ask him all of my top-of-mind AI questions including: ▪️ How OpenAI thinks about extending RL beyond code and math ▪️ The current state of alignment research as more powerful models loom ▪️ The future of continual learning ▪️ How startups should think about building their own models/harnesses And he also shared some great stories around OpenAI’s pioneering work on math. YouTube: Spotify: Apple: 0:00 Intro 1:53 Research Intern Capability Timelines 4:59 Math Breakthroughs 7:59 RL Beyond Verifiable Tasks 12:32 RL vs In-Context 19:01 Allocating Compute Internally 28:18 AI for Science 31:40 Pattern Matching 33:23 Solving the Hardest Math Problems 37:40 Chain of Thought Monitoring 44:33 Generalization and Value Alignment in Models 47:57 Inside OpenAI 51:55 Quickfire

Jacob Effron

138,475 просмотров • 5 месяцев назад

Always enjoy getting to chat with swyx 🇸🇬 AIE Singapore! on our annual cross-episode with Latent.Space on the state of AI. We hit on what’s shifted, what surprised us and what’s next. We covered: ▪️ Whether AI infrastructure has finally stabilized ▪️ Implications of agents buying developer tools ▪️ The AI coding wars ▪️ The foundation model vibe shift ▪️ Why Swyx reversed his view on open models ▪️ When to train your own model ▪️ What's top of mind for the best AI engineers YouTube: Spotify: Apple: 0:00 Intro 1:17 What the Top AI Engineers Are Thinking About 2:13 Has AI Infra Finally Stabilized? 6:39 When Does Doing RL In-House Make Sense? 11:26 Why Selling Dev Tools to Agents is Different 17:18 AI Coding Wars 29:04 Consumer AI Plateau 30:22 Codex vs Claude Code 44:52 Future of Open Models
54:52

Always enjoy getting to chat with swyx 🇸🇬 AIE Singapore! on our annual cross-episode with Latent.Space on the state of AI. We hit on what’s shifted, what surprised us and what’s next. We covered: ▪️ Whether AI infrastructure has finally stabilized ▪️ Implications of agents buying developer tools ▪️ The AI coding wars ▪️ The foundation model vibe shift ▪️ Why Swyx reversed his view on open models ▪️ When to train your own model ▪️ What's top of mind for the best AI engineers YouTube: Spotify: Apple: 0:00 Intro 1:17 What the Top AI Engineers Are Thinking About 2:13 Has AI Infra Finally Stabilized? 6:39 When Does Doing RL In-House Make Sense? 11:26 Why Selling Dev Tools to Agents is Different 17:18 AI Coding Wars 29:04 Consumer AI Plateau 30:22 Codex vs Claude Code 44:52 Future of Open Models

Jacob Effron

18,282 просмотров • 5 месяцев назад

Sonnet 4.5 & the AI Plateau Myth — epic conversation with Sholto Douglas of Anthropic 0:00 - Intro 1:09 - What's Behind The Rapid Pace of AI Releases at Anthropic 2:49 - Opus, Sonnet, and Haiku Model Tiers 4:14 - Sholto's Story: From Australian Fencer to AI Researcher 12:01 - The YouTube Effect: Mastery Through Observation 16:16 - Breaking Into AI Research Without Traditional Academic Signals 18:29 - DeepMind, Gemini, and Building Inference Stacks 23:05 - Why Anthropic? Culture and Mission Differences Amongst AI Research Labs 25:08 - What Is "Taste" in AI Research? 31:46 - This Week's Big Launch: Sonnet 4.5, Best Coding Model in the World 36:40 - From 7 Hours to 30 Hours: The Long-Running AI Agent Breakthrough 38:41 - How AI Agents Self-Correct and Maintain Coherence 43:13 - The Role of Memory in Extended Coding Sessions 47:42 - Pre-Training vs. RL: Textbooks vs. Worked Problems 52:11 - Test-Time Compute & Reinforcement Learning 55:55 - Why RL Finally Started Working on LLMs in 2024 59:38 - The Path to AGI 1:02:05 - Are We Hitting a Plateau in AI? So Many Low Hanging Fruits 1:03:41 - Beyond Coding: GDPVal & Impact Economic Sectors 1:05:47 - Preparing for 10-100x Individual Leverage & The Upcoming Robotics Explosion
1:10:03

Sonnet 4.5 & the AI Plateau Myth — epic conversation with Sholto Douglas of Anthropic 0:00 - Intro 1:09 - What's Behind The Rapid Pace of AI Releases at Anthropic 2:49 - Opus, Sonnet, and Haiku Model Tiers 4:14 - Sholto's Story: From Australian Fencer to AI Researcher 12:01 - The YouTube Effect: Mastery Through Observation 16:16 - Breaking Into AI Research Without Traditional Academic Signals 18:29 - DeepMind, Gemini, and Building Inference Stacks 23:05 - Why Anthropic? Culture and Mission Differences Amongst AI Research Labs 25:08 - What Is "Taste" in AI Research? 31:46 - This Week's Big Launch: Sonnet 4.5, Best Coding Model in the World 36:40 - From 7 Hours to 30 Hours: The Long-Running AI Agent Breakthrough 38:41 - How AI Agents Self-Correct and Maintain Coherence 43:13 - The Role of Memory in Extended Coding Sessions 47:42 - Pre-Training vs. RL: Textbooks vs. Worked Problems 52:11 - Test-Time Compute & Reinforcement Learning 55:55 - Why RL Finally Started Working on LLMs in 2024 59:38 - The Path to AGI 1:02:05 - Are We Hitting a Plateau in AI? So Many Low Hanging Fruits 1:03:41 - Beyond Coding: GDPVal & Impact Economic Sectors 1:05:47 - Preparing for 10-100x Individual Leverage & The Upcoming Robotics Explosion

Matt Turck

93,678 просмотров • 11 месяцев назад

The full Transformer vs Post-Transformer debate is live. 80 minutes. Seven rounds. No slides. Real disagreement. Lukasz Kaiser came to defend the Transformer. Adrian Kosowski, Llion Jones, and Mathias Lechner made the case for what comes next. 00:00 Contenders enter the ring 06:30 Lukasz Kaiser defends the Transformer 10:08 Adrian Kosowski on BDH and the PageRank Moment for AI 17:35 Llion Jones: Why Transformers aren't the final architecture 29:50 Mathias Lechner on Liquid AI’s approach, Fast Weights, and Self-Replacing AI 40:28 Reasoning Beyond Language 44:15 Scaling Laws: Transformer vs Post Transformer 50:31 Benchmarks, Coding Models, and Perplexity 1:04:00 Continual Learning and Dynamic Weights This is the ultimate source of truth on the subject.
1:20:46

The full Transformer vs Post-Transformer debate is live. 80 minutes. Seven rounds. No slides. Real disagreement. Lukasz Kaiser came to defend the Transformer. Adrian Kosowski, Llion Jones, and Mathias Lechner made the case for what comes next. 00:00 Contenders enter the ring 06:30 Lukasz Kaiser defends the Transformer 10:08 Adrian Kosowski on BDH and the PageRank Moment for AI 17:35 Llion Jones: Why Transformers aren't the final architecture 29:50 Mathias Lechner on Liquid AI’s approach, Fast Weights, and Self-Replacing AI 40:28 Reasoning Beyond Language 44:15 Scaling Laws: Transformer vs Post Transformer 50:31 Benchmarks, Coding Models, and Perplexity 1:04:00 Continual Learning and Dynamic Weights This is the ultimate source of truth on the subject.

Zuzanna Stamirowska

1,378,336 просмотров • 4 месяцев назад

.Jerry Tworek helped drive o1, o3, and Codex at OpenAI where he was VP of Research from 2019 to 2025. Then he left to pursue “types of research that are hard to do at OpenAI.” This week on Unsupervised Learning, I sat down with Jerry to discuss where AI research is headed and what he learned from seven years at the forefront of the field. - Why he left OpenAI after helping create some of its biggest breakthroughs - Why Jerry updated his AGI timeline after building reasoning models - The real limits of scaling reinforcement learning - Why Anthropic has done so well in coding - Inside OpenAI's pivotal decisions - What makes great AI researchers Timestamps: 0:00 Intro 1:26 Scaling Paradigms in AI 3:36 Challenges in Reinforcement Learning 11:48 AGI Timelines 18:36 Converging Labs and Economic Forces 25:05 Jerry's Departure from OpenAI 31:18 Pivotal Decisions in OpenAI's Journey 35:06 Balancing Research and Product Development 38:42 The Future of AI Coding 41:33 Specialization vs. Generalization in AI 48:47 Hiring and Building Research Teams 55:21 Quickfire Listen here: YouTube: Spotify: Apple:
1:02:59

.Jerry Tworek helped drive o1, o3, and Codex at OpenAI where he was VP of Research from 2019 to 2025. Then he left to pursue “types of research that are hard to do at OpenAI.” This week on Unsupervised Learning, I sat down with Jerry to discuss where AI research is headed and what he learned from seven years at the forefront of the field. - Why he left OpenAI after helping create some of its biggest breakthroughs - Why Jerry updated his AGI timeline after building reasoning models - The real limits of scaling reinforcement learning - Why Anthropic has done so well in coding - Inside OpenAI's pivotal decisions - What makes great AI researchers Timestamps: 0:00 Intro 1:26 Scaling Paradigms in AI 3:36 Challenges in Reinforcement Learning 11:48 AGI Timelines 18:36 Converging Labs and Economic Forces 25:05 Jerry's Departure from OpenAI 31:18 Pivotal Decisions in OpenAI's Journey 35:06 Balancing Research and Product Development 38:42 The Future of AI Coding 41:33 Specialization vs. Generalization in AI 48:47 Hiring and Building Research Teams 55:21 Quickfire Listen here: YouTube: Spotify: Apple:

Jacob Effron

43,788 просмотров • 8 месяцев назад

"As a proprietary model builder, you're kind of starting to get squeezed in.” That's Igor Babuschkin’s take on where the biggest AI labs stand today. He explains how you need the models to get way better to keep margin, but at some level they may be too sensitive to release. This week, I sat down with Igor on Unsupervised Learning. It was a fascinating conversation with someone who has real perspective on the questions everyone in AI is asking right now. From DeepMind's StarCraft project to early reasoning work at OpenAI to co-founding xAI, Igor has had a front-row seat to nearly every major AI breakthrough. He's now the co-founder of River AI, building individualized, locally-run AI models. We discuss: ▪️Why proprietary model labs might be in trouble ▪️What it's like working with Elon ▪️Building Colossus in 120 days ▪️Should enterprises train their own models ▪️What’s left for humans as models get better ▪️Why he left xAI to bet on personal local AI instead ▪️The three bets River is taking ▪️What's stopping AI from moving beyond coding ▪️Reflections on the rapid pace of the past years 0:00 Intro 1:17 Writing Fiction on Where AI Is Headed 4:46 Cracking Agents Beyond Coding 10:29 Why Igor Left to Start River 12:22 River's Three Big Bets 18:06 Weights vs. Memory: The Personalization Debate 22:04 Should Enterprises Train Their Own Models? 25:10 Are Proprietary Labs Losing Their Edge? 32:16 The China Open-Source Problem 44:19 The Elon Call That Started xAI 50:18 Thoughts on Cursor Acquisition 52:16 What's Actually Bottlenecking AI 56:55 Humans, Machines, and Staying Relevant 1:01:29 Igor's Odds This All Goes Well YouTube: Spotify: Apple:
1:04:14

"As a proprietary model builder, you're kind of starting to get squeezed in.” That's Igor Babuschkin’s take on where the biggest AI labs stand today. He explains how you need the models to get way better to keep margin, but at some level they may be too sensitive to release. This week, I sat down with Igor on Unsupervised Learning. It was a fascinating conversation with someone who has real perspective on the questions everyone in AI is asking right now. From DeepMind's StarCraft project to early reasoning work at OpenAI to co-founding xAI, Igor has had a front-row seat to nearly every major AI breakthrough. He's now the co-founder of River AI, building individualized, locally-run AI models. We discuss: ▪️Why proprietary model labs might be in trouble ▪️What it's like working with Elon ▪️Building Colossus in 120 days ▪️Should enterprises train their own models ▪️What’s left for humans as models get better ▪️Why he left xAI to bet on personal local AI instead ▪️The three bets River is taking ▪️What's stopping AI from moving beyond coding ▪️Reflections on the rapid pace of the past years 0:00 Intro 1:17 Writing Fiction on Where AI Is Headed 4:46 Cracking Agents Beyond Coding 10:29 Why Igor Left to Start River 12:22 River's Three Big Bets 18:06 Weights vs. Memory: The Personalization Debate 22:04 Should Enterprises Train Their Own Models? 25:10 Are Proprietary Labs Losing Their Edge? 32:16 The China Open-Source Problem 44:19 The Elon Call That Started xAI 50:18 Thoughts on Cursor Acquisition 52:16 What's Actually Bottlenecking AI 56:55 Humans, Machines, and Staying Relevant 1:01:29 Igor's Odds This All Goes Well YouTube: Spotify: Apple:

Jacob Effron

168,964 просмотров • 1 месяц назад

How GPT-5 thinks, with OpenAI VP of Research Jerry Tworek 00:00 - Intro 01:01 - What Reasoning Actually Means in AI 02:32 - Chain of Thought: Models Thinking in Words 05:25 - How Models Decide How Long to Think 07:24 - Evolution from o1 to o3 to GPT-5 11:00 - The Road to OpenAI: Growing up in Poland, Dropping out of School, Trading 20:32 - Working on Robotics and Rubik's Cube Solving 23:02 - A Day in the Life: Talking to Researchers 24:06 - How Research Priorities Are Determined 26:53 - OpenAI's Culture of Transparency 29:32 - Balancing Research with Shipping Fast 31:52 - Using OpenAI's Own Tools Daily 32:43 - Pre-Training Plus RL: The Modern AI Stack 35:10 - Reinforcement Learning 101: Training Dogs 40:17 - The Evolution of Deep Reinforcement Learning 42:09 - When GPT-4 Seemed Underwhelming at First 45:39 - How RLHF Made GPT-4 Actually Useful 48:02 - Unsupervised vs Supervised Learning 49:59 - GRPO and How DeepSeek Accelerated US Research 53:05 - What It Takes to Scale Reinforcement Learning 55:36 - Agentic AI and Long-Horizon Thinking 59:19 - Alignment as an RL Problem 1:01:11 - Winning ICPC World Finals Without Specific Training 1:05:53 - Applying RL Beyond Math and Coding 1:09:15 - The Path from Here to AGI 1:12:23 - Pure RL vs Language Models
1:16:04

How GPT-5 thinks, with OpenAI VP of Research Jerry Tworek 00:00 - Intro 01:01 - What Reasoning Actually Means in AI 02:32 - Chain of Thought: Models Thinking in Words 05:25 - How Models Decide How Long to Think 07:24 - Evolution from o1 to o3 to GPT-5 11:00 - The Road to OpenAI: Growing up in Poland, Dropping out of School, Trading 20:32 - Working on Robotics and Rubik's Cube Solving 23:02 - A Day in the Life: Talking to Researchers 24:06 - How Research Priorities Are Determined 26:53 - OpenAI's Culture of Transparency 29:32 - Balancing Research with Shipping Fast 31:52 - Using OpenAI's Own Tools Daily 32:43 - Pre-Training Plus RL: The Modern AI Stack 35:10 - Reinforcement Learning 101: Training Dogs 40:17 - The Evolution of Deep Reinforcement Learning 42:09 - When GPT-4 Seemed Underwhelming at First 45:39 - How RLHF Made GPT-4 Actually Useful 48:02 - Unsupervised vs Supervised Learning 49:59 - GRPO and How DeepSeek Accelerated US Research 53:05 - What It Takes to Scale Reinforcement Learning 55:36 - Agentic AI and Long-Horizon Thinking 59:19 - Alignment as an RL Problem 1:01:11 - Winning ICPC World Finals Without Specific Training 1:05:53 - Applying RL Beyond Math and Coding 1:09:15 - The Path from Here to AGI 1:12:23 - Pure RL vs Language Models

Matt Turck

451,229 просмотров • 11 месяцев назад

Why AI Can Now Make Discoveries - my conversation with Dan Roberts, Lead of the Foundations of Reinforcement Learning team at OpenAI 00:00 Intro: AI's wild week in mathematics 01:21 What OpenAI's Foundations of RL team does 03:08 Dan's journey: from black holes and quantum gravity to frontier AI 07:04 Are AI systems becoming useful for real science 08:21 The AI math moment: Erdős, OpenAI, DeepMind, and Anthropic 08:52 Why the OpenAI result was an act of exploration 10:25 OpenAI vs. DeepMind: informal reasoning vs. formal proof 12:13 RL 101: learning by doing, not just watching 15:10 Why reinforcement learning works 15:58 How RL breaks: sparse feedback and long-horizon tasks 17:03 RLHF: how human feedback shaped early language models 18:48 Move 37, self-play, and the search for novel strategies 22:16 Explore vs. exploit in scientific discovery 24:49 Why RL may now be "the cake," not the cherry on top 25:46 Why RL started working with large language models 27:29 Is RL "sucking supervision through a straw"? 28:47 Why language may be the grounding layer for intelligence 31:46 A contrarian take on the Bitter Lesson 32:41 What test-time compute actually is 34:50 How RL gives models the ability to think 35:40 Verifiable rewards, math, coding, and the messy real world 38:00 What physics can teach us about AI 42:08 Is there a thermodynamics of AI? 43:08 From Erdős problems to Einstein-level AI 45:16 Is AI already doing original science? 45:51 How far are we from AI automating AI research 47:41 Why Dan is excited about the future of science
49:06

Why AI Can Now Make Discoveries - my conversation with Dan Roberts, Lead of the Foundations of Reinforcement Learning team at OpenAI 00:00 Intro: AI's wild week in mathematics 01:21 What OpenAI's Foundations of RL team does 03:08 Dan's journey: from black holes and quantum gravity to frontier AI 07:04 Are AI systems becoming useful for real science 08:21 The AI math moment: Erdős, OpenAI, DeepMind, and Anthropic 08:52 Why the OpenAI result was an act of exploration 10:25 OpenAI vs. DeepMind: informal reasoning vs. formal proof 12:13 RL 101: learning by doing, not just watching 15:10 Why reinforcement learning works 15:58 How RL breaks: sparse feedback and long-horizon tasks 17:03 RLHF: how human feedback shaped early language models 18:48 Move 37, self-play, and the search for novel strategies 22:16 Explore vs. exploit in scientific discovery 24:49 Why RL may now be "the cake," not the cherry on top 25:46 Why RL started working with large language models 27:29 Is RL "sucking supervision through a straw"? 28:47 Why language may be the grounding layer for intelligence 31:46 A contrarian take on the Bitter Lesson 32:41 What test-time compute actually is 34:50 How RL gives models the ability to think 35:40 Verifiable rewards, math, coding, and the messy real world 38:00 What physics can teach us about AI 42:08 Is there a thermodynamics of AI? 43:08 From Erdős problems to Einstein-level AI 45:16 Is AI already doing original science? 45:51 How far are we from AI automating AI research 47:41 Why Dan is excited about the future of science

Matt Turck

69,801 просмотров • 3 месяцев назад

Thanksgiving-week treat: an epic conversation on Frontier AI with Lukasz Kaiser -co-author of “Attention Is All You Need” (Transformers) and leading research scientist at OpenAI working on GPT-5.1-era reasoning models. 00:00 – Cold open and intro 01:29 – “AI slowdown” vs a wild week of new frontier models 08:03 – Low-hanging fruit, infra, RL training and better data 11:39 – What is a reasoning model, in plain language 17:02 – Chain-of-thought and training the thinking process with RL 21:39 – Łukasz’s path: from logic and France to Google and Kurzweil 24:20 – Inside the Transformer story and what “attention” really means 28:42 – From Google Brain to OpenAI: culture, scale and GPUs 32:49 – What’s next for pre-training, GPUs and distillation 37:29 – Can we still understand these models? Circuits, sparsity and black boxes 39:42 – GPT-4 → GPT-5 → GPT-5.1: what actually changed 42:40 – Post-training, safety and teaching GPT-5.1 different tones 46:16 – How long should GPT-5.1 think? Reasoning tokens and jagged abilities 47:43 – The five-year-old’s dot puzzle that still breaks frontier models 52:22 – Generalization, child-like learning and whether reasoning is enough 53:48 – Beyond Transformers: ARC, LeCun’s ideas and multimodal bottlenecks 56:10 – GPT-5.1 Codex Max, long-running agents and compaction 1:00:06 – Will foundation models eat most apps? The translation analogy and trust 1:02:34 – What still needs to be solved, and where AI might go next
1:05:25

Thanksgiving-week treat: an epic conversation on Frontier AI with Lukasz Kaiser -co-author of “Attention Is All You Need” (Transformers) and leading research scientist at OpenAI working on GPT-5.1-era reasoning models. 00:00 – Cold open and intro 01:29 – “AI slowdown” vs a wild week of new frontier models 08:03 – Low-hanging fruit, infra, RL training and better data 11:39 – What is a reasoning model, in plain language 17:02 – Chain-of-thought and training the thinking process with RL 21:39 – Łukasz’s path: from logic and France to Google and Kurzweil 24:20 – Inside the Transformer story and what “attention” really means 28:42 – From Google Brain to OpenAI: culture, scale and GPUs 32:49 – What’s next for pre-training, GPUs and distillation 37:29 – Can we still understand these models? Circuits, sparsity and black boxes 39:42 – GPT-4 → GPT-5 → GPT-5.1: what actually changed 42:40 – Post-training, safety and teaching GPT-5.1 different tones 46:16 – How long should GPT-5.1 think? Reasoning tokens and jagged abilities 47:43 – The five-year-old’s dot puzzle that still breaks frontier models 52:22 – Generalization, child-like learning and whether reasoning is enough 53:48 – Beyond Transformers: ARC, LeCun’s ideas and multimodal bottlenecks 56:10 – GPT-5.1 Codex Max, long-running agents and compaction 1:00:06 – Will foundation models eat most apps? The translation analogy and trust 1:02:34 – What still needs to be solved, and where AI might go next

Matt Turck

168,007 просмотров • 10 месяцев назад

It’s hard to imagine more of a dream Unsupervised Learning guest than Yann LeCun. Yann is one of the godfathers of AI, and he has some fascinating contrarian views on the limitations of LLMs. It was incredible to get to have a wide-ranging discussion with Yann about these views, reflections on his time at Meta and departure and what’s next for him. We hit on: ▪️ LLM limitations and a path forward for robotics ▪️ Why he left Meta ▪️ How he came to so dramatically disagree with his Turing co-laureates Geoff Hinton and Yoshua Bengio on LLMs ▪️ His predictions for 2027 ▪️ His new company AMI and the bet on world models ▪️ Why he compares OpenAI and Anthropic to Sun Microsystems ▪️ Why he tells PhD students to stop working on LLMs Plus some sharp views on the current safety discourse, how breakthrough research actually happens and what FAIR got right and wrong. YouTube: Spotify: Apple:
1:21:56

It’s hard to imagine more of a dream Unsupervised Learning guest than Yann LeCun. Yann is one of the godfathers of AI, and he has some fascinating contrarian views on the limitations of LLMs. It was incredible to get to have a wide-ranging discussion with Yann about these views, reflections on his time at Meta and departure and what’s next for him. We hit on: ▪️ LLM limitations and a path forward for robotics ▪️ Why he left Meta ▪️ How he came to so dramatically disagree with his Turing co-laureates Geoff Hinton and Yoshua Bengio on LLMs ▪️ His predictions for 2027 ▪️ His new company AMI and the bet on world models ▪️ Why he compares OpenAI and Anthropic to Sun Microsystems ▪️ Why he tells PhD students to stop working on LLMs Plus some sharp views on the current safety discourse, how breakthrough research actually happens and what FAIR got right and wrong. YouTube: Spotify: Apple:

Jacob Effron

211,609 просмотров • 4 месяцев назад

Why AI Progress Suddenly Feels Real - my conversation with Yann Dubois, who co-leads the Post-Training Frontiers team at OpenAI 00:00 - Intro 01:30 - Why recent AI progress feels like a step function 04:13 - Model reliability & the emotional rollercoaster of shipping GPT-5.5 07:33 - How OpenAI structures vertical and horizontal teams 09:49 - Improving model efficiency and test-time compute 12:32 - Yann's journey from Switzerland to OpenAI 15:37 - Reasoning in 2026: Real-world utility vs verifiable rewards 18:34 - GPT-5.5 Thinking vs Pro: Scaling test-time compute 20:09 - How reasoning models become more efficient 23:23 - Pre-training scaling and overcoming the data wall 27:03 - Multimodal data, synthetic data, and embodied AI 31:05 - Demystifying mid-training and post-training 37:21 - Does RL create new capabilities in AI? 38:53 - The challenges and frontier of scaling RL 43:09 - Is building AI models a craft or a strict science 48:21 - How AI models generalize across different domains 54:18 - How reinforcement learning cures AI hallucinations 56:04 - Negative generalization and conflicting instructions 58:05 - Can RL scale to law, medicine, and the broader economy? 1:00:19 - The evaluation bottleneck and Model as a Judge 1:04:21 - Continuous AI progress & continual learning 1:08:49 - Will foundation models eat the agent harness 1:11:23 - Why startups should focus on the last mile of AI
1:13:56

Why AI Progress Suddenly Feels Real - my conversation with Yann Dubois, who co-leads the Post-Training Frontiers team at OpenAI 00:00 - Intro 01:30 - Why recent AI progress feels like a step function 04:13 - Model reliability & the emotional rollercoaster of shipping GPT-5.5 07:33 - How OpenAI structures vertical and horizontal teams 09:49 - Improving model efficiency and test-time compute 12:32 - Yann's journey from Switzerland to OpenAI 15:37 - Reasoning in 2026: Real-world utility vs verifiable rewards 18:34 - GPT-5.5 Thinking vs Pro: Scaling test-time compute 20:09 - How reasoning models become more efficient 23:23 - Pre-training scaling and overcoming the data wall 27:03 - Multimodal data, synthetic data, and embodied AI 31:05 - Demystifying mid-training and post-training 37:21 - Does RL create new capabilities in AI? 38:53 - The challenges and frontier of scaling RL 43:09 - Is building AI models a craft or a strict science 48:21 - How AI models generalize across different domains 54:18 - How reinforcement learning cures AI hallucinations 56:04 - Negative generalization and conflicting instructions 58:05 - Can RL scale to law, medicine, and the broader economy? 1:00:19 - The evaluation bottleneck and Model as a Judge 1:04:21 - Continuous AI progress & continual learning 1:08:49 - Will foundation models eat the agent harness 1:11:23 - Why startups should focus on the last mile of AI

Matt Turck

101,314 просмотров • 4 месяцев назад

Frontier Notes from AGI House: Jerry Tworek Jerry Tworek on why “the era of evals is done.” 𝐅𝐮𝐥𝐥 𝐢𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰 ↓ Jerry Tworek spent seven years at OpenAI, where he worked on research that helped shape modern coding and reasoning models, including Codex and reinforcement learning for reasoning. He later served as VP of Research, and is now CEO of , building toward automated AI research. At AGI House, Jerry sat down with Rocky Yu for a technical conversation on Codex, HumanEval, Copilot, RL, tool use, test-time compute, and automated AI labs. We cover: › Why code became the first serious LLM vertical › What HumanEval taught the field about verifiable rewards › Why Jerry thinks “the era of evals is done” › How real-world deployment differs from static benchmarks › What GitHub Copilot taught OpenAI about product quality › How reinforcement learning shaped modern reasoning behaviors › Why tool use is “99% systems and 1% algorithms” › Whether test-time compute still has room to scale › What it takes to build automated AI research labs Watch the full interview below.
42:08

Frontier Notes from AGI House: Jerry Tworek Jerry Tworek on why “the era of evals is done.” 𝐅𝐮𝐥𝐥 𝐢𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰 ↓ Jerry Tworek spent seven years at OpenAI, where he worked on research that helped shape modern coding and reasoning models, including Codex and reinforcement learning for reasoning. He later served as VP of Research, and is now CEO of , building toward automated AI research. At AGI House, Jerry sat down with Rocky Yu for a technical conversation on Codex, HumanEval, Copilot, RL, tool use, test-time compute, and automated AI labs. We cover: › Why code became the first serious LLM vertical › What HumanEval taught the field about verifiable rewards › Why Jerry thinks “the era of evals is done” › How real-world deployment differs from static benchmarks › What GitHub Copilot taught OpenAI about product quality › How reinforcement learning shaped modern reasoning behaviors › Why tool use is “99% systems and 1% algorithms” › Whether test-time compute still has room to scale › What it takes to build automated AI research labs Watch the full interview below.

AGI House

12,497 просмотров • 2 месяцев назад

Are Chinese open models being oversold as reaching the frontier? We’re back with another vibe check episode with Ari Morcos and Rob Toews. These are a ton of fun to record and this was a particularly meaty one given everything going on. We hit on: ▪️Did Chinese open models catch the US frontier? ▪️How much does distillation explain China's progress ▪️Reactions to Anthropic Backlash ▪️What a Stuxnet style backdoor in model weights looks like ▪️The OpenAI and Hugging Face breach ▪️xAI and Cursor ▪️Prediction check-ins 0:00 Intro 2:41 China's Open Source Models Catch Up 7:42 Does Distillation Explain China's Rise? 13:54 The Geopolitical Risk of Chinese AI Models 20:28 Should the Government Restrict Open Models? 22:06 What Are the Labs Really Learning From You? 29:22 Future of Government Regulation 40:50 The OpenAI-Hugging Face Hack 47:59 Grok, Cursor, and the Value of Real Data 56:52 SSI and OpenRouter 1:02:52 Venture Capital's Return to Deep Tech 1:05:16 Quickfire YouTube: Spotify: Apple:
1:12:15

Are Chinese open models being oversold as reaching the frontier? We’re back with another vibe check episode with Ari Morcos and Rob Toews. These are a ton of fun to record and this was a particularly meaty one given everything going on. We hit on: ▪️Did Chinese open models catch the US frontier? ▪️How much does distillation explain China's progress ▪️Reactions to Anthropic Backlash ▪️What a Stuxnet style backdoor in model weights looks like ▪️The OpenAI and Hugging Face breach ▪️xAI and Cursor ▪️Prediction check-ins 0:00 Intro 2:41 China's Open Source Models Catch Up 7:42 Does Distillation Explain China's Rise? 13:54 The Geopolitical Risk of Chinese AI Models 20:28 Should the Government Restrict Open Models? 22:06 What Are the Labs Really Learning From You? 29:22 Future of Government Regulation 40:50 The OpenAI-Hugging Face Hack 47:59 Grok, Cursor, and the Value of Real Data 56:52 SSI and OpenRouter 1:02:52 Venture Capital's Return to Deep Tech 1:05:16 Quickfire YouTube: Spotify: Apple:

Jacob Effron

14,537 просмотров • 1 месяц назад

Deeply thoughtful conversation with Zico Kolter, board member at OpenAI and head of the machine learning department at Carnegie Mellon University, about AI safety, AI security, agents and frontier AI 00:00 Intro 01:32 OpenAI board role and Safety & Security Committee 03:53 How OpenAI reviews major model releases 05:33 OpenAI’s preparedness framework explained 09:46 Are frontier AI models getting safer? 12:33 Why AI safety does not come from scale 15:23 The four categories of AI risk 19:38 Doomerism vs accelerationism in AI 24:11 The six-month AI pause debate 26:20 AI safety as a global effort 28:04 How Zico Kolter got into machine learning 31:05 OpenAI in the early days 34:14 Why Carnegie Mellon became an AI powerhouse 38:43 What Gray Swan does in AI security 40:44 AI safety vs AI security 43:15 The GCG jailbreak paper 49:19 How AI labs responded to jailbreak research 50:19 State-of-the-art AI defenses 52:32 State-of-the-art AI attacks 54:22 Why AI agents expand the attack surface 58:39 Are AI agents ready for production? 59:40 Mechanistic interpretability explained 1:02:31 Will AI be safer in two years? 1:03:46 Reinforcement learning and self-improving models 1:08:09 Do post-transformer architectures matter 1:09:29 Best research directions in AI now 1:11:00 Zico Kolter’s Intro to Modern AI course 1:14:53 Why modern AI is simpler than people think
1:16:39

Deeply thoughtful conversation with Zico Kolter, board member at OpenAI and head of the machine learning department at Carnegie Mellon University, about AI safety, AI security, agents and frontier AI 00:00 Intro 01:32 OpenAI board role and Safety & Security Committee 03:53 How OpenAI reviews major model releases 05:33 OpenAI’s preparedness framework explained 09:46 Are frontier AI models getting safer? 12:33 Why AI safety does not come from scale 15:23 The four categories of AI risk 19:38 Doomerism vs accelerationism in AI 24:11 The six-month AI pause debate 26:20 AI safety as a global effort 28:04 How Zico Kolter got into machine learning 31:05 OpenAI in the early days 34:14 Why Carnegie Mellon became an AI powerhouse 38:43 What Gray Swan does in AI security 40:44 AI safety vs AI security 43:15 The GCG jailbreak paper 49:19 How AI labs responded to jailbreak research 50:19 State-of-the-art AI defenses 52:32 State-of-the-art AI attacks 54:22 Why AI agents expand the attack surface 58:39 Are AI agents ready for production? 59:40 Mechanistic interpretability explained 1:02:31 Will AI be safer in two years? 1:03:46 Reinforcement learning and self-improving models 1:08:09 Do post-transformer architectures matter 1:09:29 Best research directions in AI now 1:11:00 Zico Kolter’s Intro to Modern AI course 1:14:53 Why modern AI is simpler than people think

Matt Turck

33,764 просмотров • 4 месяцев назад

Here's my conversation with DHH. He is back for round 2! It was an epic fun 5 hour conversation about the future of programming, AI, Linux, and human civilization. It's here on X in full and is up everywhere else (see comment). Timestamps: 0:00 - Episode highlight 1:27 - Introduction 2:56 - Programming with AI agents 18:14 - How software will change 27:30 - AI impact on open source 37:21 - Building Omarchy Linux distro 47:05 - Vibe coding vs agentic engineering 1:00:06 - The end of manual programming 1:10:24 - Advice for programmers 1:22:30 - Surviving Internet Hate 1:31:46 - Programming setup for AI Agents 1:44:11 - Obsessing about speed 2:07:06 - Voice prompting vs typing 2:21:05 - Best AI coding models 2:37:55 - Best AI coding harnesses 2:50:57 - AI video generation and filmmaking 3:10:28 - Fatherhood 3:38:35 - Linux will win the desktop 3:49:51 - PewDiePie 3:59:24 - Future of programming 4:22:17 - Politics and immigration 4:53:54 - Longevity, over-optimization, and fear of death 5:05:38 - Eternal recurrence and future of human civization
5:15:50

Here's my conversation with DHH. He is back for round 2! It was an epic fun 5 hour conversation about the future of programming, AI, Linux, and human civilization. It's here on X in full and is up everywhere else (see comment). Timestamps: 0:00 - Episode highlight 1:27 - Introduction 2:56 - Programming with AI agents 18:14 - How software will change 27:30 - AI impact on open source 37:21 - Building Omarchy Linux distro 47:05 - Vibe coding vs agentic engineering 1:00:06 - The end of manual programming 1:10:24 - Advice for programmers 1:22:30 - Surviving Internet Hate 1:31:46 - Programming setup for AI Agents 1:44:11 - Obsessing about speed 2:07:06 - Voice prompting vs typing 2:21:05 - Best AI coding models 2:37:55 - Best AI coding harnesses 2:50:57 - AI video generation and filmmaking 3:10:28 - Fatherhood 3:38:35 - Linux will win the desktop 3:49:51 - PewDiePie 3:59:24 - Future of programming 4:22:17 - Politics and immigration 4:53:54 - Longevity, over-optimization, and fear of death 5:05:38 - Eternal recurrence and future of human civization

Lex Fridman

874,628 просмотров • 1 месяц назад

In this episode, OpenAI Chief Research Officer Mark Chen joins allen to flambé shrimp, cook Korean stew, and chat about being at the frontier of AI research: why scaling laws and pre-training still matter, how OpenAI chooses research bets and allocates compute, what it means to develop research taste, why evals are in crisis, how to avoid benchmark-maxing, and what it will take for models to handle long-horizon real-world work, multimodal reasoning, and eventually end-to-end AI research. Timestamps: 0:00 Intro 0:28 The Soup Story 1:52 From Trading to AI Research 3:21 How to Develop Research Taste 5:23 RL, Evals, and Superhuman Benchmarks 8:17 Cooking Begins on the Impulse Stove 8:53 Scaling Laws, Pre-Training, and Reasoning 12:33 OpenAI’s Research Roadmap and Compute Allocation 15:48 What Makes a Great Researcher 19:33 The Evals Crisis and Benchmark-Maxing 24:34 Jagged Intelligence, Context, and Long-Horizon Learning 27:14 Shrimp Flambé and New Research Bets 31:32 Multimodal Models and One Architecture 32:36 Vibe Researching and End-to-End AI Research 34:36 Failed Bets, Postmortems, and OpenAI’s Alpha 37:07 Final Taste Test 37:53 Overrated vs. Underrated AI Research 41:00 Closing
41:16

In this episode, OpenAI Chief Research Officer Mark Chen joins allen to flambé shrimp, cook Korean stew, and chat about being at the frontier of AI research: why scaling laws and pre-training still matter, how OpenAI chooses research bets and allocates compute, what it means to develop research taste, why evals are in crisis, how to avoid benchmark-maxing, and what it will take for models to handle long-horizon real-world work, multimodal reasoning, and eventually end-to-end AI research. Timestamps: 0:00 Intro 0:28 The Soup Story 1:52 From Trading to AI Research 3:21 How to Develop Research Taste 5:23 RL, Evals, and Superhuman Benchmarks 8:17 Cooking Begins on the Impulse Stove 8:53 Scaling Laws, Pre-Training, and Reasoning 12:33 OpenAI’s Research Roadmap and Compute Allocation 15:48 What Makes a Great Researcher 19:33 The Evals Crisis and Benchmark-Maxing 24:34 Jagged Intelligence, Context, and Long-Horizon Learning 27:14 Shrimp Flambé and New Research Bets 31:32 Multimodal Models and One Architecture 32:36 Vibe Researching and End-to-End AI Research 34:36 Failed Bets, Postmortems, and OpenAI’s Alpha 37:07 Final Taste Test 37:53 Overrated vs. Underrated AI Research 41:00 Closing

Latent.Space

62,796 просмотров • 3 месяцев назад

When Mudith Jayasekara and I met Gabe Pereyra, we were expecting just another vanilla intro call and instead had the best yarn about research, the state of LLMs, and where intelligence is actually heading. It's rare to meet a founder this deep in the weeds who's also building for one of the most important verticals in this new age of intelligence So it was awesome to sit down with Gabe for an extended discussion on what it take to build agents that can reliably complete work over hours, days, or even longer? We talked about why agents today struggle with search and long context windows and how techniques like KV-cache compaction, synthetic data, and continual learning could help. 0:00 Introduction 0:36 Getting legal agents to review the whole data room 2:08 Data rooms larger than any context window 5:28 How far open-source models can go 7:58 Where specialist models fit in legal AI 10:59 Training legal models when client data is off-limits 13:06 Teaching a model how a law firm works 13:59 What belongs in context vs. model weights 15:36 From firm-wide AI to a model for every lawyer 18:37 What training adds beyond retrieving the right cases 20:26 Why context windows have plateaued 24:01 How models could learn continuously on the job 26:12 Can AI recursively improve AI research? 27:07 Research agents can run experiments but not choose them 30:00 Why open-ended research is hard to train 33:47 Why deployment, not intelligence, is the bottleneck 35:08 The cost of frontier intelligence 36:59 Different neolabs, different paths to intelligence 39:26 Using open datasets to compare research methods 41:13 Conclusion
41:50

When Mudith Jayasekara and I met Gabe Pereyra, we were expecting just another vanilla intro call and instead had the best yarn about research, the state of LLMs, and where intelligence is actually heading. It's rare to meet a founder this deep in the weeds who's also building for one of the most important verticals in this new age of intelligence So it was awesome to sit down with Gabe for an extended discussion on what it take to build agents that can reliably complete work over hours, days, or even longer? We talked about why agents today struggle with search and long context windows and how techniques like KV-cache compaction, synthetic data, and continual learning could help. 0:00 Introduction 0:36 Getting legal agents to review the whole data room 2:08 Data rooms larger than any context window 5:28 How far open-source models can go 7:58 Where specialist models fit in legal AI 10:59 Training legal models when client data is off-limits 13:06 Teaching a model how a law firm works 13:59 What belongs in context vs. model weights 15:36 From firm-wide AI to a model for every lawyer 18:37 What training adds beyond retrieving the right cases 20:26 Why context windows have plateaued 24:01 How models could learn continuously on the job 26:12 Can AI recursively improve AI research? 27:07 Research agents can run experiments but not choose them 30:00 Why open-ended research is hard to train 33:47 Why deployment, not intelligence, is the bottleneck 35:08 The cost of frontier intelligence 36:59 Different neolabs, different paths to intelligence 39:26 Using open datasets to compare research methods 41:13 Conclusion

Charlie O'Neill

92,869 просмотров • 2 месяцев назад

Failing to Understand the Exponential, Again? My conversation with Julian Schrittwieser - Julian Schrittwieser (Anthropic, AlphaGo Zero, MuZero) - on Move 37, Scaling RL, Nobel Prize for AI, and the AI frontier: 00:00 - Cold open: “We’re not seeing any slowdown.” 00:32 - Intro — Meet Julian 01:09 - The “exponential” from inside frontier labs 04:46 - 2026–2027: agents that work a full day; expert-level breadth 08:58 - Benchmarks vs reality: long-horizon work, GDP-Val, user value 10:26 - Move 37 — what actually happened and why it mattered 13:55 - Novel science: AlphaCode/AlphaTensor → when does AI earn a Nobel? 16:25 - Discontinuity vs smooth progress (and warning signs) 19:08 - Does pre-training + RL get us there? (AGI debates aside) 20:55 - Sutton’s “RL from scratch”? Julian’s take 23:03 - Julian’s path: Google → DeepMind → Anthropic 26:45 - AlphaGo (learn + search) in plain English 30:16 - AlphaGo Zero (no human data) 31:00 - AlphaZero (one algorithm: Go, chess, shogi) 31:46 - MuZero (planning with a learned world model) 33:23 -Lessons for today’s agents: search + learning at scale 34:57 - Do LLMs already have implicit world models? 39:02 - Why RL on LLMs took time (stability, feedback loops) 41:43 - Compute & scaling for RL — what we see so far 42:35 - Rewards frontier: human prefs, rubrics, RLVR, process rewards 44:36 - RL training data & the “flywheel” (and why quality matters) 48:02 - RL & Agents 101 — why RL unlocks robustness 50:51 - Should builders use RL-as-a-service? Or just tools + prompts? 52:18 - What’s missing for dependable agents (capability vs engineering) 53:51 - Evals & Goodhart — internal vs external benchmarks 57:35 - Mechanistic interpretability & “Golden Gate Claude” 1:00:03 - Safety & alignment at Anthropic — how it shows up in practice 1:03:48 - Jobs: human–AI complementarity (comparative advantage) 1:06:33 - Inequality, policy, and the case for 10× productivity → abundance 1:09:24 - Closing thoughts
1:09:56

Failing to Understand the Exponential, Again? My conversation with Julian Schrittwieser - Julian Schrittwieser (Anthropic, AlphaGo Zero, MuZero) - on Move 37, Scaling RL, Nobel Prize for AI, and the AI frontier: 00:00 - Cold open: “We’re not seeing any slowdown.” 00:32 - Intro — Meet Julian 01:09 - The “exponential” from inside frontier labs 04:46 - 2026–2027: agents that work a full day; expert-level breadth 08:58 - Benchmarks vs reality: long-horizon work, GDP-Val, user value 10:26 - Move 37 — what actually happened and why it mattered 13:55 - Novel science: AlphaCode/AlphaTensor → when does AI earn a Nobel? 16:25 - Discontinuity vs smooth progress (and warning signs) 19:08 - Does pre-training + RL get us there? (AGI debates aside) 20:55 - Sutton’s “RL from scratch”? Julian’s take 23:03 - Julian’s path: Google → DeepMind → Anthropic 26:45 - AlphaGo (learn + search) in plain English 30:16 - AlphaGo Zero (no human data) 31:00 - AlphaZero (one algorithm: Go, chess, shogi) 31:46 - MuZero (planning with a learned world model) 33:23 -Lessons for today’s agents: search + learning at scale 34:57 - Do LLMs already have implicit world models? 39:02 - Why RL on LLMs took time (stability, feedback loops) 41:43 - Compute & scaling for RL — what we see so far 42:35 - Rewards frontier: human prefs, rubrics, RLVR, process rewards 44:36 - RL training data & the “flywheel” (and why quality matters) 48:02 - RL & Agents 101 — why RL unlocks robustness 50:51 - Should builders use RL-as-a-service? Or just tools + prompts? 52:18 - What’s missing for dependable agents (capability vs engineering) 53:51 - Evals & Goodhart — internal vs external benchmarks 57:35 - Mechanistic interpretability & “Golden Gate Claude” 1:00:03 - Safety & alignment at Anthropic — how it shows up in practice 1:03:48 - Jobs: human–AI complementarity (comparative advantage) 1:06:33 - Inequality, policy, and the case for 10× productivity → abundance 1:09:24 - Closing thoughts

Matt Turck

235,526 просмотров • 11 месяцев назад