Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Why off-the-shelf AI models aren't enough for the enterprise. Prime Intellect's will brown and Rayan discuss owning your intelligence, benchmark limitations, and multi-agent systems on The Bench. Full episode out now! 03:10 Own vs. Rent Your Intelligence 05:15 Capturing Institutional Knowledge 15:45 Why Frontier Models Fail to Do Real... show more

Vals AI

27,129 subscribers

11,165 views • 13 days ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

3 Comments

Vals AI13 days ago

You can now listen on Spotify!

Scott Shapiro13 days ago

@willcb Institutional knowledge includes the exceptions people carry in their heads. I'm curious how teams capture those without turning every employee into a full-time maintainer of the AI's instructions.

Miyuki Matthews13 days ago

@willcb Great episode!

Related Videos

Why AI Can Now Make Discoveries - my conversation with Dan Roberts, Lead of the Foundations of Reinforcement Learning team at OpenAI 00:00 Intro: AI's wild week in mathematics 01:21 What OpenAI's Foundations of RL team does 03:08 Dan's journey: from black holes and quantum gravity to frontier AI 07:04 Are AI systems becoming useful for real science 08:21 The AI math moment: Erdős, OpenAI, DeepMind, and Anthropic 08:52 Why the OpenAI result was an act of exploration 10:25 OpenAI vs. DeepMind: informal reasoning vs. formal proof 12:13 RL 101: learning by doing, not just watching 15:10 Why reinforcement learning works 15:58 How RL breaks: sparse feedback and long-horizon tasks 17:03 RLHF: how human feedback shaped early language models 18:48 Move 37, self-play, and the search for novel strategies 22:16 Explore vs. exploit in scientific discovery 24:49 Why RL may now be "the cake," not the cherry on top 25:46 Why RL started working with large language models 27:29 Is RL "sucking supervision through a straw"? 28:47 Why language may be the grounding layer for intelligence 31:46 A contrarian take on the Bitter Lesson 32:41 What test-time compute actually is 34:50 How RL gives models the ability to think 35:40 Verifiable rewards, math, coding, and the messy real world 38:00 What physics can teach us about AI 42:08 Is there a thermodynamics of AI? 43:08 From Erdős problems to Einstein-level AI 45:16 Is AI already doing original science? 45:51 How far are we from AI automating AI research 47:41 Why Dan is excited about the future of science

Why AI Can Now Make Discoveries - my conversation with Dan Roberts, Lead of the Foundations of Reinforcement Learning team at OpenAI 00:00 Intro: AI's wild week in mathematics 01:21 What OpenAI's Foundations of RL team does 03:08 Dan's journey: from black holes and quantum gravity to frontier AI 07:04 Are AI systems becoming useful for real science 08:21 The AI math moment: Erdős, OpenAI, DeepMind, and Anthropic 08:52 Why the OpenAI result was an act of exploration 10:25 OpenAI vs. DeepMind: informal reasoning vs. formal proof 12:13 RL 101: learning by doing, not just watching 15:10 Why reinforcement learning works 15:58 How RL breaks: sparse feedback and long-horizon tasks 17:03 RLHF: how human feedback shaped early language models 18:48 Move 37, self-play, and the search for novel strategies 22:16 Explore vs. exploit in scientific discovery 24:49 Why RL may now be "the cake," not the cherry on top 25:46 Why RL started working with large language models 27:29 Is RL "sucking supervision through a straw"? 28:47 Why language may be the grounding layer for intelligence 31:46 A contrarian take on the Bitter Lesson 32:41 What test-time compute actually is 34:50 How RL gives models the ability to think 35:40 Verifiable rewards, math, coding, and the messy real world 38:00 What physics can teach us about AI 42:08 Is there a thermodynamics of AI? 43:08 From Erdős problems to Einstein-level AI 45:16 Is AI already doing original science? 45:51 How far are we from AI automating AI research 47:41 Why Dan is excited about the future of science

Matt Turck

69,801 views • 4 months ago

Why AI Progress Suddenly Feels Real - my conversation with Yann Dubois, who co-leads the Post-Training Frontiers team at OpenAI 00:00 - Intro 01:30 - Why recent AI progress feels like a step function 04:13 - Model reliability & the emotional rollercoaster of shipping GPT-5.5 07:33 - How OpenAI structures vertical and horizontal teams 09:49 - Improving model efficiency and test-time compute 12:32 - Yann's journey from Switzerland to OpenAI 15:37 - Reasoning in 2026: Real-world utility vs verifiable rewards 18:34 - GPT-5.5 Thinking vs Pro: Scaling test-time compute 20:09 - How reasoning models become more efficient 23:23 - Pre-training scaling and overcoming the data wall 27:03 - Multimodal data, synthetic data, and embodied AI 31:05 - Demystifying mid-training and post-training 37:21 - Does RL create new capabilities in AI? 38:53 - The challenges and frontier of scaling RL 43:09 - Is building AI models a craft or a strict science 48:21 - How AI models generalize across different domains 54:18 - How reinforcement learning cures AI hallucinations 56:04 - Negative generalization and conflicting instructions 58:05 - Can RL scale to law, medicine, and the broader economy? 1:00:19 - The evaluation bottleneck and Model as a Judge 1:04:21 - Continuous AI progress & continual learning 1:08:49 - Will foundation models eat the agent harness 1:11:23 - Why startups should focus on the last mile of AI

Why AI Progress Suddenly Feels Real - my conversation with Yann Dubois, who co-leads the Post-Training Frontiers team at OpenAI 00:00 - Intro 01:30 - Why recent AI progress feels like a step function 04:13 - Model reliability & the emotional rollercoaster of shipping GPT-5.5 07:33 - How OpenAI structures vertical and horizontal teams 09:49 - Improving model efficiency and test-time compute 12:32 - Yann's journey from Switzerland to OpenAI 15:37 - Reasoning in 2026: Real-world utility vs verifiable rewards 18:34 - GPT-5.5 Thinking vs Pro: Scaling test-time compute 20:09 - How reasoning models become more efficient 23:23 - Pre-training scaling and overcoming the data wall 27:03 - Multimodal data, synthetic data, and embodied AI 31:05 - Demystifying mid-training and post-training 37:21 - Does RL create new capabilities in AI? 38:53 - The challenges and frontier of scaling RL 43:09 - Is building AI models a craft or a strict science 48:21 - How AI models generalize across different domains 54:18 - How reinforcement learning cures AI hallucinations 56:04 - Negative generalization and conflicting instructions 58:05 - Can RL scale to law, medicine, and the broader economy? 1:00:19 - The evaluation bottleneck and Model as a Judge 1:04:21 - Continuous AI progress & continual learning 1:08:49 - Will foundation models eat the agent harness 1:11:23 - Why startups should focus on the last mile of AI

Matt Turck

101,314 views • 4 months ago

State of Enterprise AI 2026: Aaron Levie on Tokenmaxxing, The Rise of Headless, and AI-Proofing Your Job 00:00 Intro 01:18 Silicon Valley engineering vs. everyone else 05:35 Are enterprise CIOs actually bullish on AI? 08:51 Tokenmaxxing & why your AI bill is about to explode 11:34 The myth of falling token costs and AI spend escaping IT budgets 17:37 The $5B startup hiding in AI compute 18:14 The mosaic of models inside every enterprise 21:28 Why coding works and the rest of knowledge work doesn't 25:53 The Bob and Sally problem: access control breaks agents 30:31 Will enterprise AI really take 10 years to roll out 32:24 The capability overhang: why faster models slow diffusion 34:23 Data is the bottleneck (it always was) 39:02 The rise of internal forward-deployed engineers 41:23 Why the AI doomers are wrong about jobs 43:43 Headless software is inevitable 46:14 What replaces per-seat pricing 47:37 How Box itself is going headless 49:42 How the org chart actually evolves 1:00:33 Future-proofing yourself as an enterprise employee 1:06:40 Are we all just going to work for OpenAI and Anthropic? 1:07:11 Where startups can still win as the labs move up

State of Enterprise AI 2026: Aaron Levie on Tokenmaxxing, The Rise of Headless, and AI-Proofing Your Job 00:00 Intro 01:18 Silicon Valley engineering vs. everyone else 05:35 Are enterprise CIOs actually bullish on AI? 08:51 Tokenmaxxing & why your AI bill is about to explode 11:34 The myth of falling token costs and AI spend escaping IT budgets 17:37 The $5B startup hiding in AI compute 18:14 The mosaic of models inside every enterprise 21:28 Why coding works and the rest of knowledge work doesn't 25:53 The Bob and Sally problem: access control breaks agents 30:31 Will enterprise AI really take 10 years to roll out 32:24 The capability overhang: why faster models slow diffusion 34:23 Data is the bottleneck (it always was) 39:02 The rise of internal forward-deployed engineers 41:23 Why the AI doomers are wrong about jobs 43:43 Headless software is inevitable 46:14 What replaces per-seat pricing 47:37 How Box itself is going headless 49:42 How the org chart actually evolves 1:00:33 Future-proofing yourself as an enterprise employee 1:06:40 Are we all just going to work for OpenAI and Anthropic? 1:07:11 Where startups can still win as the labs move up

Matt Turck

49,691 views • 4 months ago

Every agent needs its own computer: my conversation with Ivan Burazin, CEO of Daytona, about sandboxes and the emerging agent stack. 00:00 Intro 02:13 What is an AI agent sandbox? 03:17 Security risks of running agents locally 05:17 Stateful vs. stateless hyperscalers 07:04 The history of cloud IDEs and the end of localhost 09:45 Do all AI agents need a sandbox? 12:26 Sandbox use cases: RL evals & background agents 14:10 Unpacking the emerging AI Agent Stack 16:20 The unsolved problem of agent memory and learning 19:37 Where sandboxes fit in the agent harness 21:35 OpenAI, Anthropic, and agent SDKs 23:06 Ivan's founder journey: From CodeAnywhere to Daytona 26:59 GTM strategies and building developer communities 33:48 Why customer support is your best GTM strategy 35:34 Leveraging Twitter during the AI super cycle 40:50 The technical anatomy of a sandbox 41:53 Why fast spin-up speeds maximize GPU efficiency 46:09 Firecracker, QEMU, and isolation primitives 49:58 Why sandbox snapshots and state forking matter 51:40 Why Daytona built a custom scheduler from scratch 55:24 The challenge of long-running stateful sandboxes 58:10 The build your own sandbox trap 1:01:03 Why AI agents might trigger a global CPU shortage 1:02:46 The future of the AI Agent Stack

Every agent needs its own computer: my conversation with Ivan Burazin, CEO of Daytona, about sandboxes and the emerging agent stack. 00:00 Intro 02:13 What is an AI agent sandbox? 03:17 Security risks of running agents locally 05:17 Stateful vs. stateless hyperscalers 07:04 The history of cloud IDEs and the end of localhost 09:45 Do all AI agents need a sandbox? 12:26 Sandbox use cases: RL evals & background agents 14:10 Unpacking the emerging AI Agent Stack 16:20 The unsolved problem of agent memory and learning 19:37 Where sandboxes fit in the agent harness 21:35 OpenAI, Anthropic, and agent SDKs 23:06 Ivan's founder journey: From CodeAnywhere to Daytona 26:59 GTM strategies and building developer communities 33:48 Why customer support is your best GTM strategy 35:34 Leveraging Twitter during the AI super cycle 40:50 The technical anatomy of a sandbox 41:53 Why fast spin-up speeds maximize GPU efficiency 46:09 Firecracker, QEMU, and isolation primitives 49:58 Why sandbox snapshots and state forking matter 51:40 Why Daytona built a custom scheduler from scratch 55:24 The challenge of long-running stateful sandboxes 58:10 The build your own sandbox trap 1:01:03 Why AI agents might trigger a global CPU shortage 1:02:46 The future of the AI Agent Stack

Matt Turck

12,239 views • 5 months ago

An agent is three things: a harness, a model, and context. If you're serious about owning your intelligence, you probably want to own all three. LangChain founder Harrison Chase joined us at our Sequoia Capital Own Your Intelligence to talk about the piece that often gets the least attention: the harness. He offers a clear heuristic for when to build your own. The more out of distribution you are from what the models were trained on, the more you'll want to customize. And good technical content on how to actually measure performance with evals and langsmith. 00:00 Introduction 00:58 The three parts of an agent: harness, model, context 02:12 What a harness actually does 03:25 Customizing the core loop with middleware 04:41 Sandboxes, file systems, sub-agents, summarization 05:47 Cognitive architectures — and when you still need them 07:03 Build your own harness or use off the shelf? 08:24 In-distribution vs. out-of-distribution: the file-editing example 09:39 Why evals define what "good" means in an organization 11:04 Harbor: what an eval task actually looks like 12:11 Comparing harnesses and models on accuracy, latency, and cost 13:20 Why observability is underrated — it's usually the context 14:34 The data flywheel: traces → curation → experiments 15:42 Getting feedback through UX design and online evaluators 16:51 Demo: LangSmith Engine 19:23 Q&A: Running Engine on Engine, and "codex-ification" 20:44 Q&A: Will harnesses converge or diverge?

An agent is three things: a harness, a model, and context. If you're serious about owning your intelligence, you probably want to own all three. LangChain founder Harrison Chase joined us at our Sequoia Capital Own Your Intelligence to talk about the piece that often gets the least attention: the harness. He offers a clear heuristic for when to build your own. The more out of distribution you are from what the models were trained on, the more you'll want to customize. And good technical content on how to actually measure performance with evals and langsmith. 00:00 Introduction 00:58 The three parts of an agent: harness, model, context 02:12 What a harness actually does 03:25 Customizing the core loop with middleware 04:41 Sandboxes, file systems, sub-agents, summarization 05:47 Cognitive architectures — and when you still need them 07:03 Build your own harness or use off the shelf? 08:24 In-distribution vs. out-of-distribution: the file-editing example 09:39 Why evals define what "good" means in an organization 11:04 Harbor: what an eval task actually looks like 12:11 Comparing harnesses and models on accuracy, latency, and cost 13:20 Why observability is underrated — it's usually the context 14:34 The data flywheel: traces → curation → experiments 15:42 Getting feedback through UX design and online evaluators 16:51 Demo: LangSmith Engine 19:23 Q&A: Running Engine on Engine, and "codex-ification" 20:44 Q&A: Will harnesses converge or diverge?

Sonya Huang 🐥

77,519 views • 2 months ago

Really fun to hang again with my friend 🃏 Noam Brown (OpenAI research scientist, our first guest ever on No Priors in early 2023) to talk about the implications of large test-time compute, and what happens when models are given $10M budgets to spend on a single task. Topics: 01:23 – Why Benchmarks Are Broken 04:19 – Compute Budgets and Projections 06:48 – How Long Should Models Think? 08:01 – Benchmarkmaxxing 09:48 – Noam's Evals 12:40 – Safety (When Model Capability Scales With Spend) 16:09 – Implications For the Model Release Cycle 18:34 – Latent Model Capability 22:27 – Limits on Recursive Self-Improvement 28:38 – Large-Scale Multi-Agent Coordination 30:39 – Competition at the Frontier 33:19 – Breaking the Benchmark Grid Equilibrium 34:57 – Why Benchmarks Should be Scaled by Cost

Really fun to hang again with my friend 🃏 Noam Brown (OpenAI research scientist, our first guest ever on No Priors in early 2023) to talk about the implications of large test-time compute, and what happens when models are given $10M budgets to spend on a single task. Topics: 01:23 – Why Benchmarks Are Broken 04:19 – Compute Budgets and Projections 06:48 – How Long Should Models Think? 08:01 – Benchmarkmaxxing 09:48 – Noam's Evals 12:40 – Safety (When Model Capability Scales With Spend) 16:09 – Implications For the Model Release Cycle 18:34 – Latent Model Capability 22:27 – Limits on Recursive Self-Improvement 28:38 – Large-Scale Multi-Agent Coordination 30:39 – Competition at the Frontier 33:19 – Breaking the Benchmark Grid Equilibrium 34:57 – Why Benchmarks Should be Scaled by Cost

sarah guo

87,850 views • 3 months ago

AI INTERVIEW: OPENAI'S SECRET WEAPON AI agents are no longer just hype—they're here to revolutionize automation, Web3, and beyond. SwarmNode.ai is building a serverless AI agent platform for scalability, efficiency, and real-world impact. In this exclusive interview, he reveals how AI swarms can outperform single models, why OpenAI’s Operator is just the beginning, and how crypto is fueling AI innovation. Plus, he breaks down DeepSeek’s game-changing AI breakthrough, the future of agent monetization, and why serverless AI could be the next frontier in automation. 01:37 – From Engineering to AI: The journey into artificial intelligence. 02:43 – The GPT-3 Moment: How OpenAI’s tech pulled him in. 04:10 – AI’s Biggest Challenge: Why real-world use cases lag behind. 05:05 – OpenAI’s Operator: Why it’s “rudimentary” (for now). 06:25 – Crypto & AI: How tokens help bootstrap AI startups. 08:15 – Can You Bootstrap a Startup with a Token? The trade-offs. 09:56 – 90% of AI Token Holders Don’t Use the Product—Does It Matter? 11:18 – What is SwarmNode?: AI agents, hosted serverlessly. 14:23 – AI Swarms: Why multiple agents outperform single models. 16:08 – What is a Swarm? A simple definition of collaborative AI. 17:32 – “How Can I Make Money with AI?”: Real-world use cases. 18:41 – AI Bounties: Hiring devs to build your custom agent. 20:50 – The Future of AI Marketplaces: Monetizing pre-built agents. 23:15 – DeepSeek’s Disruption: Why it’s good news for AI. 24:46 – Is SwarmNode Compatible with DeepSeek? How it integrates. 26:17 – SwarmNode vs. AI Launchpads: What makes it different? 27:42 – Why Serverless Matters: Cost savings & efficiency. 29:53 – AI Agents in the Real World: Booking flights, managing workflows, and more. 31:11 – Building SwarmNode for Developers: Why it started as a personal project. 32:27 – Explosive Growth: 200,000 AI agent executions in 5 weeks. 34:41 – Why SwarmNode Agents Aren’t Visible on 𝕏 Yet. 36:46 – Startup Hiring Lessons: Finding top AI talent. 39:15 – Why SwarmNode is Built in Python (and What’s Next). 40:32 – Scaling AI Workloads: Handling traffic surges. 41:42 – AWS & Cost Challenges: The biggest monetization hurdle. 42:58 – 2025: The Year of Mass AI Adoption. 45:22 – Should We Be Worried About AI’s Rapid Growth? 46:46 – The Most Underrated AI Tools Right Now. 47:34 – What’s Next for SwarmNode?: Making AI accessible to everyone.

AI INTERVIEW: OPENAI'S SECRET WEAPON AI agents are no longer just hype—they're here to revolutionize automation, Web3, and beyond. SwarmNode.ai is building a serverless AI agent platform for scalability, efficiency, and real-world impact. In this exclusive interview, he reveals how AI swarms can outperform single models, why OpenAI’s Operator is just the beginning, and how crypto is fueling AI innovation. Plus, he breaks down DeepSeek’s game-changing AI breakthrough, the future of agent monetization, and why serverless AI could be the next frontier in automation. 01:37 – From Engineering to AI: The journey into artificial intelligence. 02:43 – The GPT-3 Moment: How OpenAI’s tech pulled him in. 04:10 – AI’s Biggest Challenge: Why real-world use cases lag behind. 05:05 – OpenAI’s Operator: Why it’s “rudimentary” (for now). 06:25 – Crypto & AI: How tokens help bootstrap AI startups. 08:15 – Can You Bootstrap a Startup with a Token? The trade-offs. 09:56 – 90% of AI Token Holders Don’t Use the Product—Does It Matter? 11:18 – What is SwarmNode?: AI agents, hosted serverlessly. 14:23 – AI Swarms: Why multiple agents outperform single models. 16:08 – What is a Swarm? A simple definition of collaborative AI. 17:32 – “How Can I Make Money with AI?”: Real-world use cases. 18:41 – AI Bounties: Hiring devs to build your custom agent. 20:50 – The Future of AI Marketplaces: Monetizing pre-built agents. 23:15 – DeepSeek’s Disruption: Why it’s good news for AI. 24:46 – Is SwarmNode Compatible with DeepSeek? How it integrates. 26:17 – SwarmNode vs. AI Launchpads: What makes it different? 27:42 – Why Serverless Matters: Cost savings & efficiency. 29:53 – AI Agents in the Real World: Booking flights, managing workflows, and more. 31:11 – Building SwarmNode for Developers: Why it started as a personal project. 32:27 – Explosive Growth: 200,000 AI agent executions in 5 weeks. 34:41 – Why SwarmNode Agents Aren’t Visible on 𝕏 Yet. 36:46 – Startup Hiring Lessons: Finding top AI talent. 39:15 – Why SwarmNode is Built in Python (and What’s Next). 40:32 – Scaling AI Workloads: Handling traffic surges. 41:42 – AWS & Cost Challenges: The biggest monetization hurdle. 42:58 – 2025: The Year of Mass AI Adoption. 45:22 – Should We Be Worried About AI’s Rapid Growth? 46:46 – The Most Underrated AI Tools Right Now. 47:34 – What’s Next for SwarmNode?: Making AI accessible to everyone.

Mario Nawfal

338,311 views • 1 year ago

Deeply thoughtful conversation with Zico Kolter, board member at OpenAI and head of the machine learning department at Carnegie Mellon University, about AI safety, AI security, agents and frontier AI 00:00 Intro 01:32 OpenAI board role and Safety & Security Committee 03:53 How OpenAI reviews major model releases 05:33 OpenAI’s preparedness framework explained 09:46 Are frontier AI models getting safer? 12:33 Why AI safety does not come from scale 15:23 The four categories of AI risk 19:38 Doomerism vs accelerationism in AI 24:11 The six-month AI pause debate 26:20 AI safety as a global effort 28:04 How Zico Kolter got into machine learning 31:05 OpenAI in the early days 34:14 Why Carnegie Mellon became an AI powerhouse 38:43 What Gray Swan does in AI security 40:44 AI safety vs AI security 43:15 The GCG jailbreak paper 49:19 How AI labs responded to jailbreak research 50:19 State-of-the-art AI defenses 52:32 State-of-the-art AI attacks 54:22 Why AI agents expand the attack surface 58:39 Are AI agents ready for production? 59:40 Mechanistic interpretability explained 1:02:31 Will AI be safer in two years? 1:03:46 Reinforcement learning and self-improving models 1:08:09 Do post-transformer architectures matter 1:09:29 Best research directions in AI now 1:11:00 Zico Kolter’s Intro to Modern AI course 1:14:53 Why modern AI is simpler than people think

Deeply thoughtful conversation with Zico Kolter, board member at OpenAI and head of the machine learning department at Carnegie Mellon University, about AI safety, AI security, agents and frontier AI 00:00 Intro 01:32 OpenAI board role and Safety & Security Committee 03:53 How OpenAI reviews major model releases 05:33 OpenAI’s preparedness framework explained 09:46 Are frontier AI models getting safer? 12:33 Why AI safety does not come from scale 15:23 The four categories of AI risk 19:38 Doomerism vs accelerationism in AI 24:11 The six-month AI pause debate 26:20 AI safety as a global effort 28:04 How Zico Kolter got into machine learning 31:05 OpenAI in the early days 34:14 Why Carnegie Mellon became an AI powerhouse 38:43 What Gray Swan does in AI security 40:44 AI safety vs AI security 43:15 The GCG jailbreak paper 49:19 How AI labs responded to jailbreak research 50:19 State-of-the-art AI defenses 52:32 State-of-the-art AI attacks 54:22 Why AI agents expand the attack surface 58:39 Are AI agents ready for production? 59:40 Mechanistic interpretability explained 1:02:31 Will AI be safer in two years? 1:03:46 Reinforcement learning and self-improving models 1:08:09 Do post-transformer architectures matter 1:09:29 Best research directions in AI now 1:11:00 Zico Kolter’s Intro to Modern AI course 1:14:53 Why modern AI is simpler than people think

Matt Turck

33,764 views • 5 months ago

In this episode, OpenAI Chief Research Officer Mark Chen joins allen to flambé shrimp, cook Korean stew, and chat about being at the frontier of AI research: why scaling laws and pre-training still matter, how OpenAI chooses research bets and allocates compute, what it means to develop research taste, why evals are in crisis, how to avoid benchmark-maxing, and what it will take for models to handle long-horizon real-world work, multimodal reasoning, and eventually end-to-end AI research. Timestamps: 0:00 Intro 0:28 The Soup Story 1:52 From Trading to AI Research 3:21 How to Develop Research Taste 5:23 RL, Evals, and Superhuman Benchmarks 8:17 Cooking Begins on the Impulse Stove 8:53 Scaling Laws, Pre-Training, and Reasoning 12:33 OpenAI’s Research Roadmap and Compute Allocation 15:48 What Makes a Great Researcher 19:33 The Evals Crisis and Benchmark-Maxing 24:34 Jagged Intelligence, Context, and Long-Horizon Learning 27:14 Shrimp Flambé and New Research Bets 31:32 Multimodal Models and One Architecture 32:36 Vibe Researching and End-to-End AI Research 34:36 Failed Bets, Postmortems, and OpenAI’s Alpha 37:07 Final Taste Test 37:53 Overrated vs. Underrated AI Research 41:00 Closing

In this episode, OpenAI Chief Research Officer Mark Chen joins allen to flambé shrimp, cook Korean stew, and chat about being at the frontier of AI research: why scaling laws and pre-training still matter, how OpenAI chooses research bets and allocates compute, what it means to develop research taste, why evals are in crisis, how to avoid benchmark-maxing, and what it will take for models to handle long-horizon real-world work, multimodal reasoning, and eventually end-to-end AI research. Timestamps: 0:00 Intro 0:28 The Soup Story 1:52 From Trading to AI Research 3:21 How to Develop Research Taste 5:23 RL, Evals, and Superhuman Benchmarks 8:17 Cooking Begins on the Impulse Stove 8:53 Scaling Laws, Pre-Training, and Reasoning 12:33 OpenAI’s Research Roadmap and Compute Allocation 15:48 What Makes a Great Researcher 19:33 The Evals Crisis and Benchmark-Maxing 24:34 Jagged Intelligence, Context, and Long-Horizon Learning 27:14 Shrimp Flambé and New Research Bets 31:32 Multimodal Models and One Architecture 32:36 Vibe Researching and End-to-End AI Research 34:36 Failed Bets, Postmortems, and OpenAI’s Alpha 37:07 Final Taste Test 37:53 Overrated vs. Underrated AI Research 41:00 Closing

Latent.Space

62,796 views • 3 months ago

FULL INTERVIEW: Airbnb CEO Brian Chesky joins TBPN to discuss recent earnings, founder mode, the future of consumer AI, marketing, and more. 01:04 How Airbnb got its startup energy back 03:40 Founder mode and fixing one corner of the business at a time 06:14 How Chesky would disrupt Airbnb today 07:18 Why chatbots are wrong for travel 09:37 The SaaSpocalypse and AI interfaces 12:25 Why everyone is building enterprise AI 17:05 Why consumer AI hasn’t broken out yet 18:15 Why Silicon Valley keeps chasing trends 23:08 Advice for designers in the AI era 31:43 Why different marketing works best

FULL INTERVIEW: Airbnb CEO Brian Chesky joins TBPN to discuss recent earnings, founder mode, the future of consumer AI, marketing, and more. 01:04 How Airbnb got its startup energy back 03:40 Founder mode and fixing one corner of the business at a time 06:14 How Chesky would disrupt Airbnb today 07:18 Why chatbots are wrong for travel 09:37 The SaaSpocalypse and AI interfaces 12:25 Why everyone is building enterprise AI 17:05 Why consumer AI hasn’t broken out yet 18:15 Why Silicon Valley keeps chasing trends 23:08 Advice for designers in the AI era 31:43 Why different marketing works best

TBPN

15,497 views • 5 months ago

Epiplexity, Reasoning & The "Alien" Behavior of LLMs: my conversation with Pavel Izmailov, AI researcher at Anthropic and professor at New York University. 00:00 - Intro 00:53 - Alien survival instincts: is continual learning about to create a major alignment problem? Riffing on the post by 🍓🍓🍓 03:33 - Did AI learn deception from sci-fi literature? 05:55 - Defining Alignment, Superalignment & OpenAI teams 08:12 - Pavel’s journey: From Russian math to OpenAI Superalignment 10:46 - Culture check: OpenAI vs. Anthropic vs. Academia 11:54 - Why move to NYU? Academia and the need for exploratory research 13:09 - Does reasoning make AI alignment harder or easier? 14:22 - Sandbagging: When models pretend to be dumb 16:19 - Scalable Oversight: Using AI to supervise AI 18:04 - Weak-to-Strong Generalization 22:43 - Mechanistic Interpretability: Inside the black box 25:08 - The reasoning explosion 27:07 - Are Transformers enough or do we need a new paradigm? 28:29 - RL vs. Test-Time Compute: What’s actually driving progress? 30:10 - Long-horizon tasks: Agents running for hours 31:49 - Epiplexity: A new theory of data information content - Pavel's work with Marc Finzi, Zico Kolter, Andrew Gordon Wilson, Yiding Jiang and Shikai Qiu. 38:29 - 2026 Predictions: Multi-agent systems & reasoning limits 39:28 - AI & Scientific progress 41:42 - Advice for PhD students

Epiplexity, Reasoning & The "Alien" Behavior of LLMs: my conversation with Pavel Izmailov, AI researcher at Anthropic and professor at New York University. 00:00 - Intro 00:53 - Alien survival instincts: is continual learning about to create a major alignment problem? Riffing on the post by 🍓🍓🍓 03:33 - Did AI learn deception from sci-fi literature? 05:55 - Defining Alignment, Superalignment & OpenAI teams 08:12 - Pavel’s journey: From Russian math to OpenAI Superalignment 10:46 - Culture check: OpenAI vs. Anthropic vs. Academia 11:54 - Why move to NYU? Academia and the need for exploratory research 13:09 - Does reasoning make AI alignment harder or easier? 14:22 - Sandbagging: When models pretend to be dumb 16:19 - Scalable Oversight: Using AI to supervise AI 18:04 - Weak-to-Strong Generalization 22:43 - Mechanistic Interpretability: Inside the black box 25:08 - The reasoning explosion 27:07 - Are Transformers enough or do we need a new paradigm? 28:29 - RL vs. Test-Time Compute: What’s actually driving progress? 30:10 - Long-horizon tasks: Agents running for hours 31:49 - Epiplexity: A new theory of data information content - Pavel's work with Marc Finzi, Zico Kolter, Andrew Gordon Wilson, Yiding Jiang and Shikai Qiu. 38:29 - 2026 Predictions: Multi-agent systems & reasoning limits 39:28 - AI & Scientific progress 41:42 - Advice for PhD students

Matt Turck

23,288 views • 9 months ago

FULL INTERVIEW: Ramez Naam says AI's self-improvement loop is 5 to 10 times too weak to take off on its own, even with no humans in the loop. OpenAI researchers used 124x more tokens per person and ran just 1.6x more experiments. He says the labs' own data doesn't match what their leaders are claiming. Ramez Naam, founder and managing partner of Planetary VC, made the case on Noahpinion that the intelligence explosion isn't close yet. He joined Theo Jaffee and sof 𓋹 to lay out the numbers behind it: 00:58 where's the intelligence explosion 03:08 124x more tokens, 7x more code, 1.6x more experiments 03:54 what a self-sustaining loop would actually take 05:11 why the labs' claims don't match their own system cards 09:48 AI capex vs big tech's free cash flow 11:20 why AI revenue growth will likely slow 14:03 why hyper-persuasion might not be possible 15:21 a teenager's 1,000 miles vs Waymo's 10 million 16:39 the math result that could break the loop open 18:28 why TasteVal doesn't measure real research taste 23:11 the one AI regulation he wants 24:39 why agent swarms have the worst scaling we've seen 27:24 Opus 5.5 stalling at 10 agents, and Columbus, Ohio vs Einstein

FULL INTERVIEW: Ramez Naam says AI's self-improvement loop is 5 to 10 times too weak to take off on its own, even with no humans in the loop. OpenAI researchers used 124x more tokens per person and ran just 1.6x more experiments. He says the labs' own data doesn't match what their leaders are claiming. Ramez Naam, founder and managing partner of Planetary VC, made the case on Noahpinion that the intelligence explosion isn't close yet. He joined Theo Jaffee and sof 𓋹 to lay out the numbers behind it: 00:58 where's the intelligence explosion 03:08 124x more tokens, 7x more code, 1.6x more experiments 03:54 what a self-sustaining loop would actually take 05:11 why the labs' claims don't match their own system cards 09:48 AI capex vs big tech's free cash flow 11:20 why AI revenue growth will likely slow 14:03 why hyper-persuasion might not be possible 15:21 a teenager's 1,000 miles vs Waymo's 10 million 16:39 the math result that could break the loop open 18:28 why TasteVal doesn't measure real research taste 23:11 the one AI regulation he wants 24:39 why agent swarms have the worst scaling we've seen 27:24 Opus 5.5 stalling at 10 agents, and Columbus, Ohio vs Einstein

MTS

88,739 views • 3 days ago

"You should really treat having your own agent as having your own child. It's not one-click automation, you're raising it." Demi Guo (Demi Guo ), CEO of Pika on why agents need taste, not just prompts. Lin Qiao (Lin Qiao), CEO of Fireworks AI on why 95% of the world's data is still locked inside enterprises and why open-source models are converging with frontier. They join @jason on This Week in AI Episode 9: 00:00 Intro to Lin Qiao and Demi Guo 08:43 95% of private data locked in enterprises 16:03 The interface for creation is a human-like agent 22:07 The death of middle management 28:03 Taste, judgment, and why "slop" is the real risk 33:40 Agents aren't tools, they're children you raise 39:36 How close is open-source to frontier? 53:04 Agents as self-expression 63:17 Meta's Muse model goes closed-source

"You should really treat having your own agent as having your own child. It's not one-click automation, you're raising it." Demi Guo (Demi Guo ), CEO of Pika on why agents need taste, not just prompts. Lin Qiao (Lin Qiao), CEO of Fireworks AI on why 95% of the world's data is still locked inside enterprises and why open-source models are converging with frontier. They join @jason on This Week in AI Episode 9: 00:00 Intro to Lin Qiao and Demi Guo 08:43 95% of private data locked in enterprises 16:03 The interface for creation is a human-like agent 22:07 The death of middle management 28:03 Taste, judgment, and why "slop" is the real risk 33:40 Agents aren't tools, they're children you raise 39:36 How close is open-source to frontier? 53:04 Agents as self-expression 63:17 Meta's Muse model goes closed-source

This Week in AI

48,202 views • 6 months ago

I went to Miami to chat with dax, co-founder of OpenCode. We talked about the future of software engineering, coding agents, and why open source matters more now than ever. Timestamps: 0:00 Intro 5:30 Miami vs San Francisco tech scene 15:05 OpenCode origin story, scaling while open-source 25:03 OpenCode vs. Anthropic: owning models, open-source AI 33:36 AI hardware shortages, predicting the future 42:15 The bet of open-weight models, China vs. US 48:34 Why inference is hard, economics of intelligence 55:36 Will developers be automated? Software engineering as a craft 1:11:02 Advice to founders, building in public, marketing I had so much fun making this with Lan. Enjoy!

I went to Miami to chat with dax, co-founder of OpenCode. We talked about the future of software engineering, coding agents, and why open source matters more now than ever. Timestamps: 0:00 Intro 5:30 Miami vs San Francisco tech scene 15:05 OpenCode origin story, scaling while open-source 25:03 OpenCode vs. Anthropic: owning models, open-source AI 33:36 AI hardware shortages, predicting the future 42:15 The bet of open-weight models, China vs. US 48:34 Why inference is hard, economics of intelligence 55:36 Will developers be automated? Software engineering as a craft 1:11:02 Advice to founders, building in public, marketing I had so much fun making this with Lan. Enjoy!

Madison Kanna

271,518 views • 3 months ago

A model is only as good as its data, and we’ve long since exhausted the internet. From here on out, model progress is gated by data production. ’s Brendan (can/do) joined us at our Sovereign AI event to talk about how RL environments get built, and why your data might be your real moat: 00:00 Introduction 00:47 A short history of the data market: crowdsourcing to agentic data 02:29 What an RL environment is: worlds, apps, tasks 03:57 Why only humans can measure the frontier 05:35 Building verifiers is the hard part 06:44 Walkthrough: a real legal RL environment 08:18 Leaderboards — and what open weights change 09:45 Post-training results on Apex Agents 11:17 Three ways companies buy data 12:49 Q&A: How do you price data? 14:17 Q&A: What "data quality" actually means 16:42 Q&A: The misunderstanding about synthetic data 18:17 Q&A: Why RL environments now — and what comes after 21:20 Q&A: Can you scale rubric generation with models? 23:00 Q&A: RL environments for cyber defense 25:33 Q&A: Build data in-house or partner?

A model is only as good as its data, and we’ve long since exhausted the internet. From here on out, model progress is gated by data production. ’s Brendan (can/do) joined us at our Sovereign AI event to talk about how RL environments get built, and why your data might be your real moat: 00:00 Introduction 00:47 A short history of the data market: crowdsourcing to agentic data 02:29 What an RL environment is: worlds, apps, tasks 03:57 Why only humans can measure the frontier 05:35 Building verifiers is the hard part 06:44 Walkthrough: a real legal RL environment 08:18 Leaderboards — and what open weights change 09:45 Post-training results on Apex Agents 11:17 Three ways companies buy data 12:49 Q&A: How do you price data? 14:17 Q&A: What "data quality" actually means 16:42 Q&A: The misunderstanding about synthetic data 18:17 Q&A: Why RL environments now — and what comes after 21:20 Q&A: Can you scale rubric generation with models? 23:00 Q&A: RL environments for cyber defense 25:33 Q&A: Build data in-house or partner?

Pat Grady

66,691 views • 2 months ago

How GPT-5 thinks, with OpenAI VP of Research Jerry Tworek 00:00 - Intro 01:01 - What Reasoning Actually Means in AI 02:32 - Chain of Thought: Models Thinking in Words 05:25 - How Models Decide How Long to Think 07:24 - Evolution from o1 to o3 to GPT-5 11:00 - The Road to OpenAI: Growing up in Poland, Dropping out of School, Trading 20:32 - Working on Robotics and Rubik's Cube Solving 23:02 - A Day in the Life: Talking to Researchers 24:06 - How Research Priorities Are Determined 26:53 - OpenAI's Culture of Transparency 29:32 - Balancing Research with Shipping Fast 31:52 - Using OpenAI's Own Tools Daily 32:43 - Pre-Training Plus RL: The Modern AI Stack 35:10 - Reinforcement Learning 101: Training Dogs 40:17 - The Evolution of Deep Reinforcement Learning 42:09 - When GPT-4 Seemed Underwhelming at First 45:39 - How RLHF Made GPT-4 Actually Useful 48:02 - Unsupervised vs Supervised Learning 49:59 - GRPO and How DeepSeek Accelerated US Research 53:05 - What It Takes to Scale Reinforcement Learning 55:36 - Agentic AI and Long-Horizon Thinking 59:19 - Alignment as an RL Problem 1:01:11 - Winning ICPC World Finals Without Specific Training 1:05:53 - Applying RL Beyond Math and Coding 1:09:15 - The Path from Here to AGI 1:12:23 - Pure RL vs Language Models

How GPT-5 thinks, with OpenAI VP of Research Jerry Tworek 00:00 - Intro 01:01 - What Reasoning Actually Means in AI 02:32 - Chain of Thought: Models Thinking in Words 05:25 - How Models Decide How Long to Think 07:24 - Evolution from o1 to o3 to GPT-5 11:00 - The Road to OpenAI: Growing up in Poland, Dropping out of School, Trading 20:32 - Working on Robotics and Rubik's Cube Solving 23:02 - A Day in the Life: Talking to Researchers 24:06 - How Research Priorities Are Determined 26:53 - OpenAI's Culture of Transparency 29:32 - Balancing Research with Shipping Fast 31:52 - Using OpenAI's Own Tools Daily 32:43 - Pre-Training Plus RL: The Modern AI Stack 35:10 - Reinforcement Learning 101: Training Dogs 40:17 - The Evolution of Deep Reinforcement Learning 42:09 - When GPT-4 Seemed Underwhelming at First 45:39 - How RLHF Made GPT-4 Actually Useful 48:02 - Unsupervised vs Supervised Learning 49:59 - GRPO and How DeepSeek Accelerated US Research 53:05 - What It Takes to Scale Reinforcement Learning 55:36 - Agentic AI and Long-Horizon Thinking 59:19 - Alignment as an RL Problem 1:01:11 - Winning ICPC World Finals Without Specific Training 1:05:53 - Applying RL Beyond Math and Coding 1:09:15 - The Path from Here to AGI 1:12:23 - Pure RL vs Language Models

Matt Turck

451,229 views • 1 year ago

Amjad Masad and Marc Andreessen on AI agents, AGI, Creativity, and Reasoning Agents are getting better, fast. Replit Agent v1 could do only 2 minutes of coherent work. v2 could do 20 minutes. Today’s v3? 200 minutes. As agents get more and more reliable, we can make huge progress in coding without ever achieving “true” AGI. In this episode, Marc Andreessen and Erik Torenberg sit down with Replit ⠕ co-founder and CEO Amjad Masad to discuss the upward trajectory of agents, how RL unlocked reasoning for modern models, why verification loops changed everything, and whether LLMs are hitting diminishing returns. 00:00 Intro 03:00 The vision behind Replit 07:00 Building apps with AI agents 09:30 When the agent becomes the programmer 13:45 Reinforcement learning and problem solving 17:30 The verification loop and multi-agent systems 26:00 Why coding is advancing faster than other Fields 33:45 AGI debate 41:15 Functional AGI and automating labor 53:10 Creativity, reasoning, and finding truth in AI 57:30 The origins of Replit 01:03:00 Hacking his university and getting caught Amjad Masad Marc Andreessen 🇺🇸 Erik Torenberg

Amjad Masad and Marc Andreessen on AI agents, AGI, Creativity, and Reasoning Agents are getting better, fast. Replit Agent v1 could do only 2 minutes of coherent work. v2 could do 20 minutes. Today’s v3? 200 minutes. As agents get more and more reliable, we can make huge progress in coding without ever achieving “true” AGI. In this episode, Marc Andreessen and Erik Torenberg sit down with Replit ⠕ co-founder and CEO Amjad Masad to discuss the upward trajectory of agents, how RL unlocked reasoning for modern models, why verification loops changed everything, and whether LLMs are hitting diminishing returns. 00:00 Intro 03:00 The vision behind Replit 07:00 Building apps with AI agents 09:30 When the agent becomes the programmer 13:45 Reinforcement learning and problem solving 17:30 The verification loop and multi-agent systems 26:00 Why coding is advancing faster than other Fields 33:45 AGI debate 41:15 Functional AGI and automating labor 53:10 Creativity, reasoning, and finding truth in AI 57:30 The origins of Replit 01:03:00 Hacking his university and getting caught Amjad Masad Marc Andreessen 🇺🇸 Erik Torenberg

a16z

741,578 views • 11 months ago

Founder and CEO of The Bot Company Kyle Vogt—who also cofounded Twitch and Cruise—shares a pint and discusses robots doing your household chores, the autonomy industry, Tesla vs Waymo, and how future $100b companies will look different. Timestamps: 00:00 Intro 00:38 The Bot Company pitch 02:05 Single-task vs. multi-task robots 04:27 What is the Turing test for robotics? 05:52 Why this time is different for home robots 08:42 The last mile in robotics and self-driving 09:47 Viral demos and hype cycles 10:38 Commercializing frontier tech 13:06 Self-driving CapEx 14:15 Regulatory hurdles 16:18 Tesla vs. Waymo 19:21 Why Kyle regrets selling Cruise 21:39 The next $100 billion company Podcast on Spotify/YouTube/etc too!

Founder and CEO of The Bot Company Kyle Vogt—who also cofounded Twitch and Cruise—shares a pint and discusses robots doing your household chores, the autonomy industry, Tesla vs Waymo, and how future $100b companies will look different. Timestamps: 00:00 Intro 00:38 The Bot Company pitch 02:05 Single-task vs. multi-task robots 04:27 What is the Turing test for robotics? 05:52 Why this time is different for home robots 08:42 The last mile in robotics and self-driving 09:47 Viral demos and hype cycles 10:38 Commercializing frontier tech 13:06 Self-driving CapEx 14:15 Regulatory hurdles 16:18 Tesla vs. Waymo 19:21 Why Kyle regrets selling Cruise 21:39 The next $100 billion company Podcast on Spotify/YouTube/etc too!

John Collison

104,632 views • 1 year ago

capability != learning new benchtalks with Parth Asawa on continual learning, where we discuss teaching models to learn from experience, measuring learning ability, the bet on parametric models, and more 01:06 What is continual learning? 04:10 Why capability and learning are different 06:13 Why build a benchmark? 08:07 Continual Learning Bench launch and reception 09:13 Anthropic's Fable release and Continual Learning Bench 11:02 How to design tasks for continual learning 18:41 The gain metric 24:01 What good looks like on the leaderboard 29:13 Failure modes: why models can't update their beliefs 31:12 Parametric systems and future architectures 34:30 Open science and AI safety 45:42 Lightning round 49:08 How to contribute to Continual Learning Bench

capability != learning new benchtalks with Parth Asawa on continual learning, where we discuss teaching models to learn from experience, measuring learning ability, the bet on parametric models, and more 01:06 What is continual learning? 04:10 Why capability and learning are different 06:13 Why build a benchmark? 08:07 Continual Learning Bench launch and reception 09:13 Anthropic's Fable release and Continual Learning Bench 11:02 How to design tasks for continual learning 18:41 The gain metric 24:01 What good looks like on the leaderboard 29:13 Failure modes: why models can't update their beliefs 31:12 Parametric systems and future architectures 34:30 Open science and AI safety 45:42 Lightning round 49:08 How to contribute to Continual Learning Bench

vincent sunn chen

41,663 views • 3 months ago