Loading video...

Video Failed to Load

Go Home

Today's AI models train once. We don't work that way. We learn continuously, forget what doesn't matter, and retain what does. That gap is what Dan Biderman and Jessy Lin are closing at Engram. AI that never stops learning, with memory that lives inside the model instead of bolted...

sequoia's profile picture

Sequoia Capital

782,351 subscribers

82,070 views • 21 days ago •via X (Twitter)

Science & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Here's my conversation with Charan Ranganath, psychologist & neuroscientist at UC Davis, specializing in human memory. We talk about memory, imagination, deja vu, false memories, why we remember, why we forget, and a lot of other fascinating mysteries about the human mind. It's here on X in full, and is up on YouTube, Spotify, and everywhere else. Links in comment. Timestamps: 0:00 - Introduction 1:03 - Experiencing self vs remembering self 14:44 - Creating memories 24:16 - Why we forget 31:53 - Training memory 42:22 - Memory hacks 54:10 - Imagination vs memory 1:03:29 - Memory competitions 1:13:18 - Science of memory 1:28:33 - Discoveries 1:39:37 - Deja vu 1:44:54 - False memories 2:04:59 - False confessions 2:08:45 - Heartbreak 2:16:19 - Nature of time 2:24:00 - Brain–computer interface (BCI) 2:38:04 - AI and memory 2:48:18 - ADHD 2:55:15 - Music 3:05:00 - Human mind
3:10:40

Here's my conversation with Charan Ranganath, psychologist & neuroscientist at UC Davis, specializing in human memory. We talk about memory, imagination, deja vu, false memories, why we remember, why we forget, and a lot of other fascinating mysteries about the human mind. It's here on X in full, and is up on YouTube, Spotify, and everywhere else. Links in comment. Timestamps: 0:00 - Introduction 1:03 - Experiencing self vs remembering self 14:44 - Creating memories 24:16 - Why we forget 31:53 - Training memory 42:22 - Memory hacks 54:10 - Imagination vs memory 1:03:29 - Memory competitions 1:13:18 - Science of memory 1:28:33 - Discoveries 1:39:37 - Deja vu 1:44:54 - False memories 2:04:59 - False confessions 2:08:45 - Heartbreak 2:16:19 - Nature of time 2:24:00 - Brain–computer interface (BCI) 2:38:04 - AI and memory 2:48:18 - ADHD 2:55:15 - Music 3:05:00 - Human mind

Lex Fridman

294,521 views • 2 years ago

Why AI Progress Suddenly Feels Real - my conversation with Yann Dubois, who co-leads the Post-Training Frontiers team at OpenAI 00:00 - Intro 01:30 - Why recent AI progress feels like a step function 04:13 - Model reliability & the emotional rollercoaster of shipping GPT-5.5 07:33 - How OpenAI structures vertical and horizontal teams 09:49 - Improving model efficiency and test-time compute 12:32 - Yann's journey from Switzerland to OpenAI 15:37 - Reasoning in 2026: Real-world utility vs verifiable rewards 18:34 - GPT-5.5 Thinking vs Pro: Scaling test-time compute 20:09 - How reasoning models become more efficient 23:23 - Pre-training scaling and overcoming the data wall 27:03 - Multimodal data, synthetic data, and embodied AI 31:05 - Demystifying mid-training and post-training 37:21 - Does RL create new capabilities in AI? 38:53 - The challenges and frontier of scaling RL 43:09 - Is building AI models a craft or a strict science 48:21 - How AI models generalize across different domains 54:18 - How reinforcement learning cures AI hallucinations 56:04 - Negative generalization and conflicting instructions 58:05 - Can RL scale to law, medicine, and the broader economy? 1:00:19 - The evaluation bottleneck and Model as a Judge 1:04:21 - Continuous AI progress & continual learning 1:08:49 - Will foundation models eat the agent harness 1:11:23 - Why startups should focus on the last mile of AI
1:13:56

Why AI Progress Suddenly Feels Real - my conversation with Yann Dubois, who co-leads the Post-Training Frontiers team at OpenAI 00:00 - Intro 01:30 - Why recent AI progress feels like a step function 04:13 - Model reliability & the emotional rollercoaster of shipping GPT-5.5 07:33 - How OpenAI structures vertical and horizontal teams 09:49 - Improving model efficiency and test-time compute 12:32 - Yann's journey from Switzerland to OpenAI 15:37 - Reasoning in 2026: Real-world utility vs verifiable rewards 18:34 - GPT-5.5 Thinking vs Pro: Scaling test-time compute 20:09 - How reasoning models become more efficient 23:23 - Pre-training scaling and overcoming the data wall 27:03 - Multimodal data, synthetic data, and embodied AI 31:05 - Demystifying mid-training and post-training 37:21 - Does RL create new capabilities in AI? 38:53 - The challenges and frontier of scaling RL 43:09 - Is building AI models a craft or a strict science 48:21 - How AI models generalize across different domains 54:18 - How reinforcement learning cures AI hallucinations 56:04 - Negative generalization and conflicting instructions 58:05 - Can RL scale to law, medicine, and the broader economy? 1:00:19 - The evaluation bottleneck and Model as a Judge 1:04:21 - Continuous AI progress & continual learning 1:08:49 - Will foundation models eat the agent harness 1:11:23 - Why startups should focus on the last mile of AI

Matt Turck

100,493 views • 1 month ago

A lot of people are calling Hermes Agent the end of OpenClaw. BRUH! It's not... Nous Research trains actual models and they built an agent around that expertise. The local model routing is solid, but the part that matters for your business is that your conversations become fine-tuning data. You can train a model on how you actually work. 00:00 The Problem with Local AI Models 00:25 Introduction to Nous Research 01:04 Cross-Platform Agent Capabilities 01:44 Deep Local Model Integration 02:30 Routing Tasks to Different Models 03:06 Conversations as Training Data 03:50 Hermes Agent vs. OpenClaw 04:15 Future Plans and Series Overview
4:45

A lot of people are calling Hermes Agent the end of OpenClaw. BRUH! It's not... Nous Research trains actual models and they built an agent around that expertise. The local model routing is solid, but the part that matters for your business is that your conversations become fine-tuning data. You can train a model on how you actually work. 00:00 The Problem with Local AI Models 00:25 Introduction to Nous Research 01:04 Cross-Platform Agent Capabilities 01:44 Deep Local Model Integration 02:30 Routing Tasks to Different Models 03:06 Conversations as Training Data 03:50 Hermes Agent vs. OpenClaw 04:15 Future Plans and Series Overview

Ray Fernando

42,398 views • 3 months ago

Stanford Frontier Systems '26, Session 3 (Full lecture) The Future of Voice Systems with @matiii from ElevenLabs 00:00 Welcome and Intro 01:31 Origin Story on Discord 05:15 The Dubbing Problem 07:44 Pipeline and Early Pivot 12:38 Building the First Model 15:24 Compute Costs and Patents 17:34 Roadmap Through 2025 22:00 Cascaded vs Fused Agents 30:38 Collaboration Over Competition 35:05 Revenue Growth and Team Design 37:56 Predictable Deployment Engine 42:32 Voice Safety and Watermarking 44:27 Research Bottlenecks Personalization 46:24 Training Tradeoffs Cascade vs Fuse 48:20 Five Year Vision Platform 51:08 Impact Work ALS and Ukraine 54:40 China Distillation and Openness 59:24 Studios AI Voice Economics 01:03:04 On Device Models and Platform Gap 01:04:36 Enterprise Tooling and Wrap Up
1:06:10

Stanford Frontier Systems '26, Session 3 (Full lecture) The Future of Voice Systems with @matiii from ElevenLabs 00:00 Welcome and Intro 01:31 Origin Story on Discord 05:15 The Dubbing Problem 07:44 Pipeline and Early Pivot 12:38 Building the First Model 15:24 Compute Costs and Patents 17:34 Roadmap Through 2025 22:00 Cascaded vs Fused Agents 30:38 Collaboration Over Competition 35:05 Revenue Growth and Team Design 37:56 Predictable Deployment Engine 42:32 Voice Safety and Watermarking 44:27 Research Bottlenecks Personalization 46:24 Training Tradeoffs Cascade vs Fuse 48:20 Five Year Vision Platform 51:08 Impact Work ALS and Ukraine 54:40 China Distillation and Openness 59:24 Studios AI Voice Economics 01:03:04 On Device Models and Platform Gap 01:04:36 Enterprise Tooling and Wrap Up

Anjney Midha

79,195 views • 3 months ago

Most people experience AI through apps. Those apps run on infrastructure owned by a handful of companies. Ben Fielding and Harry Grieve of gensyn on why that needs to change, and what it's going to take to rebuild AI as crypto-native open infrastructure. 00:00 – Intro 00:29 – Misconceptions about AI infrastructure 01:20 – Why centralization in AI is a deeper problem than people realize 04:19 – Why AI needs crypto 05:51 – How AI models are trained 08:15 – The rise of autonomous AI agents with onchain identities 10:37 – Lightning round
17:23

Most people experience AI through apps. Those apps run on infrastructure owned by a handful of companies. Ben Fielding and Harry Grieve of gensyn on why that needs to change, and what it's going to take to rebuild AI as crypto-native open infrastructure. 00:00 – Intro 00:29 – Misconceptions about AI infrastructure 01:20 – Why centralization in AI is a deeper problem than people realize 04:19 – Why AI needs crypto 05:51 – How AI models are trained 08:15 – The rise of autonomous AI agents with onchain identities 10:37 – Lightning round

a16z crypto

24,109 views • 2 months ago

*Major* open source AI drop today. Can America win the Open AI race? My conversation with Nathan Lambert and Luca Soldaini 🎀 of Ai2 about the launch of Olmo 3 00:00 – Cold Open 00:39 – Welcome & today’s big announcement 01:18 – Introducing the Olmo 3 model family 02:07 – What “base models” really are (and why they matter) 05:51 – Dolma 3: the data behind Olmo 3 08:06 – Performance vs Qwen, Gemma, DeepSeek 10:28 – What true open source means (and why it’s rare) 12:51 – Intermediate checkpoints, transparency, and why AI2 publishes everything 16:37 – Why Qwen is everywhere (including U.S. startups) 18:31 – Why Chinese labs go open source (and why U.S. labs don’t) 20:28 – Inside ATOM: the U.S. response to China’s model surge 22:13 – The rise of “thinking models” and inference-time scaling 35:58 – The full Olmo pipeline, explained simply 46:52 – Pre-training: data, scale, and avoiding catastrophic spikes 50:27 – Mid-training (tail patching) and avoiding test leakage 52:06 – Why long-context training matters 55:28 – SFT: building the foundation for reasoning 1:04:53 – Preference tuning & why DPO still works 1:10:51 – The hard part: RLVR, long reasoning chains, and infrastructure pain 1:13:59 – Why RL is so technically brutal 1:18:17 – Complexity tax vs AGI hype 1:21:58 – How everyone can contribute to the future of AI 1:27:26 – Closing thoughts
1:28:10

*Major* open source AI drop today. Can America win the Open AI race? My conversation with Nathan Lambert and Luca Soldaini 🎀 of Ai2 about the launch of Olmo 3 00:00 – Cold Open 00:39 – Welcome & today’s big announcement 01:18 – Introducing the Olmo 3 model family 02:07 – What “base models” really are (and why they matter) 05:51 – Dolma 3: the data behind Olmo 3 08:06 – Performance vs Qwen, Gemma, DeepSeek 10:28 – What true open source means (and why it’s rare) 12:51 – Intermediate checkpoints, transparency, and why AI2 publishes everything 16:37 – Why Qwen is everywhere (including U.S. startups) 18:31 – Why Chinese labs go open source (and why U.S. labs don’t) 20:28 – Inside ATOM: the U.S. response to China’s model surge 22:13 – The rise of “thinking models” and inference-time scaling 35:58 – The full Olmo pipeline, explained simply 46:52 – Pre-training: data, scale, and avoiding catastrophic spikes 50:27 – Mid-training (tail patching) and avoiding test leakage 52:06 – Why long-context training matters 55:28 – SFT: building the foundation for reasoning 1:04:53 – Preference tuning & why DPO still works 1:10:51 – The hard part: RLVR, long reasoning chains, and infrastructure pain 1:13:59 – Why RL is so technically brutal 1:18:17 – Complexity tax vs AGI hype 1:21:58 – How everyone can contribute to the future of AI 1:27:26 – Closing thoughts

Matt Turck

37,482 views • 7 months ago

My conversation with OpenAI co-founder Greg Brockman This is the most detailed first-person account of the 72 hours after Sam Altman was fired. We also go deep on what comes next: the global race to AGI, why ChatGPT stopped showing reasoning, how much of OpenAI's own code is now written by AI ("it's hard to know what percent is not"), and the untold story of how OpenAI actually started in 2015. 00:00:00 Introduction 00:00:49 Meeting Sam Altman and Starting OpenAI 00:02:40 Building the Founding Team 00:04:25 DeepMind's Lead Over OpenAI 00:04:54 Changing OpenAI to a For-Profit Model 00:06:05 Breakthrough Moments at OpenAI 00:08:22 What Dota 2 Meant for OpenAI 00:10:04 Reasoning Versus Prediction 00:11:59 Tensions Grow at OpenAI 00:15:44 Sam Altman's Firing 00:17:49 Greg Quits OpenAI 00:19:56 Sam Explores Deal with Microsoft's Satya 00:20:28 Petition for Altman's Return 00:23:43 Ilya Sutskever Leaves OpenAI 00:24:59 Lessons Learned after Sam Ousting 00:28:22 The Thing Ilya Said that Greg Can't Forget 00:32:22 Is AI Going Parabolic? 00:33:24 How Much of OpenAI's Code is Written by AI? 00:36:21 Do AI Chatbots Tell Us What We Want to Hear? 00:38:06 The Global AI Race to Reach AGI 00:38:40 What Happens if US Doesn't Reach AGI First? 00:39:49 Are Countries Stealing AI Advancements? 00:40:38 Why ChatGPT No Longer Shows Reasoning 00:41:47 The Finite Constraints of Compute 00:43:38 On Investing Early in Data Centers 00:46:31 The Future of Data Center Specialization 00:47:52 How to Decide Whose Queries to Serve 00:49:08 OpenAI on Consumer vs Enterprise Models 00:53:05 Data Centers in Space? 01:00:56 What Should AI Regulation Look Like? 01:04:33 The Future of AI-Powered Entrepreneurship 01:04:44 AI and Job Loss 01:07:15 The Skills Young People Should Invest In 01:11:30 What Does Success Look Like For You? Full episode on X below. Also find it on: • YouTube: • Spotify: • Apple:
1:11:41

My conversation with OpenAI co-founder Greg Brockman This is the most detailed first-person account of the 72 hours after Sam Altman was fired. We also go deep on what comes next: the global race to AGI, why ChatGPT stopped showing reasoning, how much of OpenAI's own code is now written by AI ("it's hard to know what percent is not"), and the untold story of how OpenAI actually started in 2015. 00:00:00 Introduction 00:00:49 Meeting Sam Altman and Starting OpenAI 00:02:40 Building the Founding Team 00:04:25 DeepMind's Lead Over OpenAI 00:04:54 Changing OpenAI to a For-Profit Model 00:06:05 Breakthrough Moments at OpenAI 00:08:22 What Dota 2 Meant for OpenAI 00:10:04 Reasoning Versus Prediction 00:11:59 Tensions Grow at OpenAI 00:15:44 Sam Altman's Firing 00:17:49 Greg Quits OpenAI 00:19:56 Sam Explores Deal with Microsoft's Satya 00:20:28 Petition for Altman's Return 00:23:43 Ilya Sutskever Leaves OpenAI 00:24:59 Lessons Learned after Sam Ousting 00:28:22 The Thing Ilya Said that Greg Can't Forget 00:32:22 Is AI Going Parabolic? 00:33:24 How Much of OpenAI's Code is Written by AI? 00:36:21 Do AI Chatbots Tell Us What We Want to Hear? 00:38:06 The Global AI Race to Reach AGI 00:38:40 What Happens if US Doesn't Reach AGI First? 00:39:49 Are Countries Stealing AI Advancements? 00:40:38 Why ChatGPT No Longer Shows Reasoning 00:41:47 The Finite Constraints of Compute 00:43:38 On Investing Early in Data Centers 00:46:31 The Future of Data Center Specialization 00:47:52 How to Decide Whose Queries to Serve 00:49:08 OpenAI on Consumer vs Enterprise Models 00:53:05 Data Centers in Space? 01:00:56 What Should AI Regulation Look Like? 01:04:33 The Future of AI-Powered Entrepreneurship 01:04:44 AI and Job Loss 01:07:15 The Skills Young People Should Invest In 01:11:30 What Does Success Look Like For You? Full episode on X below. Also find it on: • YouTube: • Spotify: • Apple:

Shane Parrish

450,952 views • 2 months ago

1/ BREAKING – DePIN's Biggest New Deal: Valeo x NATIX Network with Alireza Ghods Host: Connor Lovely 🎬 We discuss: - Why a $3B, Fortune 500 company wants Natix data - What is a World Foundational Model? - Why autonomous driving needs synthetic data Timestamps: 00:00 - Introduction 01:14 - Updates on VX360 and Data Sales 02:09 - Vision Language Model and Its Applications 04:39 - Importance of Diverse Data for World Models 06:20 - Details on the Valeo Deal 08:15 - Use Cases and Future Plans for World Models 10:45 - Synthetic Data and Real-World Applications 19:37 - Future of Physical AI and Autonomous Driving 23:51 - Conclusion Watch on other platforms below ⬇️
25:05

1/ BREAKING – DePIN's Biggest New Deal: Valeo x NATIX Network with Alireza Ghods Host: Connor Lovely 🎬 We discuss: - Why a $3B, Fortune 500 company wants Natix data - What is a World Foundational Model? - Why autonomous driving needs synthetic data Timestamps: 00:00 - Introduction 01:14 - Updates on VX360 and Data Sales 02:09 - Vision Language Model and Its Applications 04:39 - Importance of Diverse Data for World Models 06:20 - Details on the Valeo Deal 08:15 - Use Cases and Future Plans for World Models 10:45 - Synthetic Data and Real-World Applications 19:37 - Future of Physical AI and Autonomous Driving 23:51 - Conclusion Watch on other platforms below ⬇️

Proof of Coverage | Tech’s Creative Agency

11,852 views • 5 months ago

Why AI Can Now Make Discoveries - my conversation with Dan Roberts, Lead of the Foundations of Reinforcement Learning team at OpenAI 00:00 Intro: AI's wild week in mathematics 01:21 What OpenAI's Foundations of RL team does 03:08 Dan's journey: from black holes and quantum gravity to frontier AI 07:04 Are AI systems becoming useful for real science 08:21 The AI math moment: Erdős, OpenAI, DeepMind, and Anthropic 08:52 Why the OpenAI result was an act of exploration 10:25 OpenAI vs. DeepMind: informal reasoning vs. formal proof 12:13 RL 101: learning by doing, not just watching 15:10 Why reinforcement learning works 15:58 How RL breaks: sparse feedback and long-horizon tasks 17:03 RLHF: how human feedback shaped early language models 18:48 Move 37, self-play, and the search for novel strategies 22:16 Explore vs. exploit in scientific discovery 24:49 Why RL may now be "the cake," not the cherry on top 25:46 Why RL started working with large language models 27:29 Is RL "sucking supervision through a straw"? 28:47 Why language may be the grounding layer for intelligence 31:46 A contrarian take on the Bitter Lesson 32:41 What test-time compute actually is 34:50 How RL gives models the ability to think 35:40 Verifiable rewards, math, coding, and the messy real world 38:00 What physics can teach us about AI 42:08 Is there a thermodynamics of AI? 43:08 From Erdős problems to Einstein-level AI 45:16 Is AI already doing original science? 45:51 How far are we from AI automating AI research 47:41 Why Dan is excited about the future of science
49:06

Why AI Can Now Make Discoveries - my conversation with Dan Roberts, Lead of the Foundations of Reinforcement Learning team at OpenAI 00:00 Intro: AI's wild week in mathematics 01:21 What OpenAI's Foundations of RL team does 03:08 Dan's journey: from black holes and quantum gravity to frontier AI 07:04 Are AI systems becoming useful for real science 08:21 The AI math moment: Erdős, OpenAI, DeepMind, and Anthropic 08:52 Why the OpenAI result was an act of exploration 10:25 OpenAI vs. DeepMind: informal reasoning vs. formal proof 12:13 RL 101: learning by doing, not just watching 15:10 Why reinforcement learning works 15:58 How RL breaks: sparse feedback and long-horizon tasks 17:03 RLHF: how human feedback shaped early language models 18:48 Move 37, self-play, and the search for novel strategies 22:16 Explore vs. exploit in scientific discovery 24:49 Why RL may now be "the cake," not the cherry on top 25:46 Why RL started working with large language models 27:29 Is RL "sucking supervision through a straw"? 28:47 Why language may be the grounding layer for intelligence 31:46 A contrarian take on the Bitter Lesson 32:41 What test-time compute actually is 34:50 How RL gives models the ability to think 35:40 Verifiable rewards, math, coding, and the messy real world 38:00 What physics can teach us about AI 42:08 Is there a thermodynamics of AI? 43:08 From Erdős problems to Einstein-level AI 45:16 Is AI already doing original science? 45:51 How far are we from AI automating AI research 47:41 Why Dan is excited about the future of science

Matt Turck

64,952 views • 1 month ago

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering
27:58

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering

Madison Kanna

119,268 views • 3 months ago

Here's my conversation with Lucas Atkins and the team at Arcee.ai on their path to training and releasing Trinity Large today. From going all in on open models built end to end in the US 6 months ago to having the model in hand is no easy feet. I loved this conversation on how to design a startup around open models and take a bold step to scale it up. I'm openly an Arcee fan, watching them take risk and pull it off. We discuss: - The state (and future) of open vs. closed models, - The business of selling open models for on-prem deployments, - The story of Arcee AI & going “all-in” on this training run, - The ATOM project, - Building frontier model training teams in 6 months, - and other great topics. I really loved this one, and think you well too. Chapters: 00:00:00 Intro: Arcee AI, Trinity Models & Trinity Large 00:08:26 Transitioning a Company to Pre-training 00:13:00 Technical Decisions: Muon and MoE 00:18:41 Scaling and MoE Training Pain 00:23:14 Post-training and RL Strategies 00:28:09 Team Structure and Data Scaling 00:31:31 The Trinity Manifesto: US Open Weights 00:42:31 Specialized Models and Distillation 00:47:12 Infrastructure and Hosting 400B 00:50:53 Open Source as a Business Moat 00:56:31 Predictions: Best Model in 2026 01:02:29 Lightning Round & Conclusions More great open model builder podcasts coming soon!
1:12:15

Here's my conversation with Lucas Atkins and the team at Arcee.ai on their path to training and releasing Trinity Large today. From going all in on open models built end to end in the US 6 months ago to having the model in hand is no easy feet. I loved this conversation on how to design a startup around open models and take a bold step to scale it up. I'm openly an Arcee fan, watching them take risk and pull it off. We discuss: - The state (and future) of open vs. closed models, - The business of selling open models for on-prem deployments, - The story of Arcee AI & going “all-in” on this training run, - The ATOM project, - Building frontier model training teams in 6 months, - and other great topics. I really loved this one, and think you well too. Chapters: 00:00:00 Intro: Arcee AI, Trinity Models & Trinity Large 00:08:26 Transitioning a Company to Pre-training 00:13:00 Technical Decisions: Muon and MoE 00:18:41 Scaling and MoE Training Pain 00:23:14 Post-training and RL Strategies 00:28:09 Team Structure and Data Scaling 00:31:31 The Trinity Manifesto: US Open Weights 00:42:31 Specialized Models and Distillation 00:47:12 Infrastructure and Hosting 400B 00:50:53 Open Source as a Business Moat 00:56:31 Predictions: Best Model in 2026 01:02:29 Lightning Round & Conclusions More great open model builder podcasts coming soon!

Nathan Lambert

26,872 views • 5 months ago

Yoshua Bengio thinks he knows how to make provably safe superintelligent agents. Bengio built the foundations of modern AI and is the most cited living scientist. He believes his alternative training setup would: 1. Guarantee honesty 2. Prevent unintended goals 3. Produce capable agents 4. Port over most data and techniques from current LLMs 5. Not be inherently more expensive, and perhaps be more intelligent Bengio claims the honesty and lack of unintended goals can be proven mathematically, at least given particular assumptions. And his new organization, LawZero, is aiming to build a scrappy prototype as soon as possible. The architecture is called 'Scientist AI' and it's based on training a model to explain empirical observations, including what people say, rather than training AIs that mimic human behaviour or seek our approval. (Bengio's frank assessment is that "reinforcement learning is evil" and that allowing AIs to independently train their successors is "the most crazy, dangerous bet that unfortunately we are on track to do.") But skeptics question whether Scientist AI really does solve the fundamental problem of 'eliciting latent knowledge' from AI models. And with the commercial race for superintelligence so intense, it's not clear whether the proposal will be able to compete or have time to bear fruit, even if it's sound in theory. On The 80,000 Hours Podcast, links below – enjoy! • Making AI honest and safe (00:00:00) • Scientist AI in plain English (00:02:27) • How Scientist AI differs from LLMs (00:06:32) • How the training data works (00:14:02) • Can this become an agent? (00:21:02) • Why Yoshua is now more optimistic (00:32:11) • Why companies can’t stop racing (00:36:35) • A working prototype won't take long (00:49:15) • Scientist models might be more capable (00:53:34) • “Reinforcement learning is evil” (01:01:27) • Scientist AI from guardrail to agent (01:08:37) • Can safe AI still be competent? (01:12:38) • How much will this cost? (01:19:29) • Can it generalise beyond maths and science? (01:23:26) • A multi-national push for superintelligence (01:39:19) • Want to work with or fund Yoshua? (01:51:16) • Why smart people ignore AI risk (01:54:45) • Don’t let AI build the next AI (02:01:33) • Why politicians miss the real risks (02:12:28) • Why Yoshua changed his mind about AI risk (02:21:27)
2:35:26

Yoshua Bengio thinks he knows how to make provably safe superintelligent agents. Bengio built the foundations of modern AI and is the most cited living scientist. He believes his alternative training setup would: 1. Guarantee honesty 2. Prevent unintended goals 3. Produce capable agents 4. Port over most data and techniques from current LLMs 5. Not be inherently more expensive, and perhaps be more intelligent Bengio claims the honesty and lack of unintended goals can be proven mathematically, at least given particular assumptions. And his new organization, LawZero, is aiming to build a scrappy prototype as soon as possible. The architecture is called 'Scientist AI' and it's based on training a model to explain empirical observations, including what people say, rather than training AIs that mimic human behaviour or seek our approval. (Bengio's frank assessment is that "reinforcement learning is evil" and that allowing AIs to independently train their successors is "the most crazy, dangerous bet that unfortunately we are on track to do.") But skeptics question whether Scientist AI really does solve the fundamental problem of 'eliciting latent knowledge' from AI models. And with the commercial race for superintelligence so intense, it's not clear whether the proposal will be able to compete or have time to bear fruit, even if it's sound in theory. On The 80,000 Hours Podcast, links below – enjoy! • Making AI honest and safe (00:00:00) • Scientist AI in plain English (00:02:27) • How Scientist AI differs from LLMs (00:06:32) • How the training data works (00:14:02) • Can this become an agent? (00:21:02) • Why Yoshua is now more optimistic (00:32:11) • Why companies can’t stop racing (00:36:35) • A working prototype won't take long (00:49:15) • Scientist models might be more capable (00:53:34) • “Reinforcement learning is evil” (01:01:27) • Scientist AI from guardrail to agent (01:08:37) • Can safe AI still be competent? (01:12:38) • How much will this cost? (01:19:29) • Can it generalise beyond maths and science? (01:23:26) • A multi-national push for superintelligence (01:39:19) • Want to work with or fund Yoshua? (01:51:16) • Why smart people ignore AI risk (01:54:45) • Don’t let AI build the next AI (02:01:33) • Why politicians miss the real risks (02:12:28) • Why Yoshua changed his mind about AI risk (02:21:27)

Rob Wiblin

65,088 views • 2 months ago

OpenAI recently released its first open-weights model since GPT-2, entering a field led by DeepSeek and Alibaba's Qwen. Ankit () breaks down these top OSS models, including what sets them apart under the hood: mixture-of-experts, long-context training, and post-training techniques that shape reasoning and alignment—and how different design choices lead to surprisingly similar performance. 00:00 – OpenAI OSS Launch 01:00 – Comparing Open Source LLM Architectures 01:46 – GPT OSS Overview 02:37 – Under The Hood of GPT OSS 03:25 – Qwen-3 Architecture 04:17 – Qwen-3 Training 05:12 – Qwen-3 Post-Training 06:08 – Qwen-3 Reasoning & RL Innovations 06:52 – DeepSeek V3 Overview 07:40 – DeepSeek V3.1 Updates 08:39 – Attention Mechanism (MLA) 09:39 – Comparing Model Sizes 10:35 – Long Context Strategies 11:25 – Reflections on Methods 12:00 – Takeaways
12:31

OpenAI recently released its first open-weights model since GPT-2, entering a field led by DeepSeek and Alibaba's Qwen. Ankit () breaks down these top OSS models, including what sets them apart under the hood: mixture-of-experts, long-context training, and post-training techniques that shape reasoning and alignment—and how different design choices lead to surprisingly similar performance. 00:00 – OpenAI OSS Launch 01:00 – Comparing Open Source LLM Architectures 01:46 – GPT OSS Overview 02:37 – Under The Hood of GPT OSS 03:25 – Qwen-3 Architecture 04:17 – Qwen-3 Training 05:12 – Qwen-3 Post-Training 06:08 – Qwen-3 Reasoning & RL Innovations 06:52 – DeepSeek V3 Overview 07:40 – DeepSeek V3.1 Updates 08:39 – Attention Mechanism (MLA) 09:39 – Comparing Model Sizes 10:35 – Long Context Strategies 11:25 – Reflections on Methods 12:00 – Takeaways

Y Combinator

208,680 views • 10 months ago

Thanksgiving-week treat: an epic conversation on Frontier AI with Lukasz Kaiser -co-author of “Attention Is All You Need” (Transformers) and leading research scientist at OpenAI working on GPT-5.1-era reasoning models. 00:00 – Cold open and intro 01:29 – “AI slowdown” vs a wild week of new frontier models 08:03 – Low-hanging fruit, infra, RL training and better data 11:39 – What is a reasoning model, in plain language 17:02 – Chain-of-thought and training the thinking process with RL 21:39 – Łukasz’s path: from logic and France to Google and Kurzweil 24:20 – Inside the Transformer story and what “attention” really means 28:42 – From Google Brain to OpenAI: culture, scale and GPUs 32:49 – What’s next for pre-training, GPUs and distillation 37:29 – Can we still understand these models? Circuits, sparsity and black boxes 39:42 – GPT-4 → GPT-5 → GPT-5.1: what actually changed 42:40 – Post-training, safety and teaching GPT-5.1 different tones 46:16 – How long should GPT-5.1 think? Reasoning tokens and jagged abilities 47:43 – The five-year-old’s dot puzzle that still breaks frontier models 52:22 – Generalization, child-like learning and whether reasoning is enough 53:48 – Beyond Transformers: ARC, LeCun’s ideas and multimodal bottlenecks 56:10 – GPT-5.1 Codex Max, long-running agents and compaction 1:00:06 – Will foundation models eat most apps? The translation analogy and trust 1:02:34 – What still needs to be solved, and where AI might go next
1:05:25

Thanksgiving-week treat: an epic conversation on Frontier AI with Lukasz Kaiser -co-author of “Attention Is All You Need” (Transformers) and leading research scientist at OpenAI working on GPT-5.1-era reasoning models. 00:00 – Cold open and intro 01:29 – “AI slowdown” vs a wild week of new frontier models 08:03 – Low-hanging fruit, infra, RL training and better data 11:39 – What is a reasoning model, in plain language 17:02 – Chain-of-thought and training the thinking process with RL 21:39 – Łukasz’s path: from logic and France to Google and Kurzweil 24:20 – Inside the Transformer story and what “attention” really means 28:42 – From Google Brain to OpenAI: culture, scale and GPUs 32:49 – What’s next for pre-training, GPUs and distillation 37:29 – Can we still understand these models? Circuits, sparsity and black boxes 39:42 – GPT-4 → GPT-5 → GPT-5.1: what actually changed 42:40 – Post-training, safety and teaching GPT-5.1 different tones 46:16 – How long should GPT-5.1 think? Reasoning tokens and jagged abilities 47:43 – The five-year-old’s dot puzzle that still breaks frontier models 52:22 – Generalization, child-like learning and whether reasoning is enough 53:48 – Beyond Transformers: ARC, LeCun’s ideas and multimodal bottlenecks 56:10 – GPT-5.1 Codex Max, long-running agents and compaction 1:00:06 – Will foundation models eat most apps? The translation analogy and trust 1:02:34 – What still needs to be solved, and where AI might go next

Matt Turck

167,926 views • 7 months ago

Core Memory podcast with Suzanne Gildert on the hunt for AI consciousness in the quantum realm. The theory being that LLMs are too constrained to really get to robots and AIs that are really like us 00:00:00 – Consciousness: The First AI Training Model? 00:02:03 – Exploring the Quantum Mind with AI 00:04:42 – Will Robots Ever Have Rights? 00:07:54 – The AI Paradox: Fear vs. Progress 00:13:20 – Can Language Models Teach Robots? 00:17:29 – Why Today’s AI Feels Like a Zombie 00:23:47 – Quantum Computing Meets Artificial Consciousness 00:29:50 – Free Will, Entanglement, and Decision Making 00:38:21 – Teaching a Robot to Think, Not Just React 00:52:51 – Will Conscious AI Power Space Exploration?
1:10:42

Core Memory podcast with Suzanne Gildert on the hunt for AI consciousness in the quantum realm. The theory being that LLMs are too constrained to really get to robots and AIs that are really like us 00:00:00 – Consciousness: The First AI Training Model? 00:02:03 – Exploring the Quantum Mind with AI 00:04:42 – Will Robots Ever Have Rights? 00:07:54 – The AI Paradox: Fear vs. Progress 00:13:20 – Can Language Models Teach Robots? 00:17:29 – Why Today’s AI Feels Like a Zombie 00:23:47 – Quantum Computing Meets Artificial Consciousness 00:29:50 – Free Will, Entanglement, and Decision Making 00:38:21 – Teaching a Robot to Think, Not Just React 00:52:51 – Will Conscious AI Power Space Exploration?

Ashlee Vance

17,189 views • 1 year ago

Gemini 3, scaling laws and the 'finite data' era: my conversation with Sebastian Borgeaud, research engineer at Google DeepMind and a pre-training lead for Gemini 3 00:00 – Cold intro: “We’re ahead of schedule” + AI is now a system 00:58 – Oriol Vinyals's “secret recipe”: better pre- + post-training 02:09 – Why AI progress still isn’t slowing down 03:04 – Are models actually getting smarter? 04:36 – Two–three years out: what changes first? 06:34 – AI doing AI research: faster, not automated 07:45 – Frontier labs: same playbook or different bets? 10:19 – Post-transformers: will a disruption happen? 10:51 – DeepMind’s advantage: research × engineering × infra 12:26 – What a Gemini 3 pre-training lead actually does 13:59 – From Europe to Cambridge to DeepMind 18:06 – Why he left RL for real-world data 20:05 – From Gopher to Chinchilla to RETRO (and why it matters) 20:28 – “Research taste”: integrate or slow everyone down 23:00 – Fixes vs moonshots: how they balance the pipeline 24:37 – Research vs product pressure (and org structure) 26:24 – Gemini 3 under the hood: MoE in plain English 28:30 – Native multimodality: the hidden costs 30:03 – Scaling laws aren’t dead (but scale isn’t everything) 33:07 – Synthetic data: powerful, dangerous? 35:00 – Reasoning traces: what he can’t say (and why) 37:18 – Long context + attention: what’s next 38:40 – Retrieval vs RAG vs long context 41:49 – The real boss fight: evals (and contamination) 42:28 – Alignment: pre-training vs post-training 43:32 – Deep Think + agents + “vibe coding” 46:34 – Continual learning: updating models over time 49:35 – Advice for researchers + founders 53:35 – “No end in sight” for progress + closing
54:56

Gemini 3, scaling laws and the 'finite data' era: my conversation with Sebastian Borgeaud, research engineer at Google DeepMind and a pre-training lead for Gemini 3 00:00 – Cold intro: “We’re ahead of schedule” + AI is now a system 00:58 – Oriol Vinyals's “secret recipe”: better pre- + post-training 02:09 – Why AI progress still isn’t slowing down 03:04 – Are models actually getting smarter? 04:36 – Two–three years out: what changes first? 06:34 – AI doing AI research: faster, not automated 07:45 – Frontier labs: same playbook or different bets? 10:19 – Post-transformers: will a disruption happen? 10:51 – DeepMind’s advantage: research × engineering × infra 12:26 – What a Gemini 3 pre-training lead actually does 13:59 – From Europe to Cambridge to DeepMind 18:06 – Why he left RL for real-world data 20:05 – From Gopher to Chinchilla to RETRO (and why it matters) 20:28 – “Research taste”: integrate or slow everyone down 23:00 – Fixes vs moonshots: how they balance the pipeline 24:37 – Research vs product pressure (and org structure) 26:24 – Gemini 3 under the hood: MoE in plain English 28:30 – Native multimodality: the hidden costs 30:03 – Scaling laws aren’t dead (but scale isn’t everything) 33:07 – Synthetic data: powerful, dangerous? 35:00 – Reasoning traces: what he can’t say (and why) 37:18 – Long context + attention: what’s next 38:40 – Retrieval vs RAG vs long context 41:49 – The real boss fight: evals (and contamination) 42:28 – Alignment: pre-training vs post-training 43:32 – Deep Think + agents + “vibe coding” 46:34 – Continual learning: updating models over time 49:35 – Advice for researchers + founders 53:35 – “No end in sight” for progress + closing

Matt Turck

51,317 views • 6 months ago

BREAKING: AI has eaten the Internet. Data labeling is so over. & $30 trillion of human work is on the verge of automation. Inside The $2.2B AI Research Accelerator, Turing Founder & CEO, Jonathan Siddharth (), joins Sourcery to break down the severe power shift in AI training: from commodity data labeling → expert research Positioning Turing apart from AI data providers like Scale AI, Mercor, & Surge. (00:00) AI Ate The Internet (00:49) Training Superintelligence: the race to AGI (02:31) Viral tweet (03:24) What Turing actually does (04:43) The internet data is “used up” — where will new data come from? (05:34) Four pillars of superintelligence: multimodality, reasoning, tool use, coding (06:07) Automating $30T of global knowledge work (09:18) The $1B revenue opportunity (10:59) Why Turing is a research-first accelerator, not a data labeler (13:45) Jonathan’s Stanford AI Lab roots & founding DNA (17:57) How models are built: pre-training vs. post-training (20:14) RLHF, reinforcement learning, & “breaking the models” (25:19) GPT-5 and the myth of rapid takeoff (30:46) Safety debates and human-in-the-loop systems (34:53) Closing Enterprise Gap: finance, insurance, & pharma (39:23) Why proprietary enterprise data is the next moat in AI
49:29

BREAKING: AI has eaten the Internet. Data labeling is so over. & $30 trillion of human work is on the verge of automation. Inside The $2.2B AI Research Accelerator, Turing Founder & CEO, Jonathan Siddharth (), joins Sourcery to break down the severe power shift in AI training: from commodity data labeling → expert research Positioning Turing apart from AI data providers like Scale AI, Mercor, & Surge. (00:00) AI Ate The Internet (00:49) Training Superintelligence: the race to AGI (02:31) Viral tweet (03:24) What Turing actually does (04:43) The internet data is “used up” — where will new data come from? (05:34) Four pillars of superintelligence: multimodality, reasoning, tool use, coding (06:07) Automating $30T of global knowledge work (09:18) The $1B revenue opportunity (10:59) Why Turing is a research-first accelerator, not a data labeler (13:45) Jonathan’s Stanford AI Lab roots & founding DNA (17:57) How models are built: pre-training vs. post-training (20:14) RLHF, reinforcement learning, & “breaking the models” (25:19) GPT-5 and the myth of rapid takeoff (30:46) Safety debates and human-in-the-loop systems (34:53) Closing Enterprise Gap: finance, insurance, & pharma (39:23) Why proprietary enterprise data is the next moat in AI

Molly O’Shea

69,971 views • 9 months ago

State of AI compute 2026: my conversation with stephen balaban of Lambda on the neocloud boom, data centers, GPUs and what's ahead 00:00 — Cold open 01:21 — Why GPU compute was never a commodity 02:45 — The H100 price index and what it gets wrong 04:02 — The real moat: technology or financing? 05:57 — Winner-take-all, or room for many neoclouds 06:48 — Are we overbuilding or underbuilding AI compute? 09:26 — What if AI gets 10x more compute-efficient? 10:44 — The real bottleneck: land, power, and shell 11:38 — The backlash against data centers — and the misinformation 15:00 — Opening the hood: from photons to tokens 17:11 — Extracting more value from the same chip 19:26 — Frontier inference and distributed training, explained 23:26 — What actually drives compute cost 25:21 — Lambda's chip stack and the NVIDIA relationship 26:17 — A multi-silicon world? CUDA, CUDNN, and NVIDIA's real moat 28:59 — Networking, storage, and the one-click cluster 34:46 — Renting vs. owning, and full vertical integration 36:24 — How global is Lambda? Does location still matter? 38:44 — The financing stack: off-take agreements, SPVs, and credit 41:16 — Why a 2023 GPU leases for more today 42:36 — A futures market for compute? 43:54 — Origin story: facial recognition, Perceptio, and Apple 47:03 — The Lambda hat and Dream Scope 48:59 — The $60K bet that became a cloud business 52:00 — Holding the team together through the hard times 54:30 — Bringing on a new CEO; Stephen as CTO 57:33 — Matching xAI on high-velocity deployment 59:29 — "AI won't write software — it will become the software" 01:01:30 — Neural software vs. vibe coding 01:04:25 — Do agents change the compute layer 01:06:14 — Self-assembling software inside Lambda 01:08:18 — Gigawatt-scale AI factories 01:08:57 — One person, one GPU 01:12:04 — Hot takes: overrated and underrated in AI
1:14:27

State of AI compute 2026: my conversation with stephen balaban of Lambda on the neocloud boom, data centers, GPUs and what's ahead 00:00 — Cold open 01:21 — Why GPU compute was never a commodity 02:45 — The H100 price index and what it gets wrong 04:02 — The real moat: technology or financing? 05:57 — Winner-take-all, or room for many neoclouds 06:48 — Are we overbuilding or underbuilding AI compute? 09:26 — What if AI gets 10x more compute-efficient? 10:44 — The real bottleneck: land, power, and shell 11:38 — The backlash against data centers — and the misinformation 15:00 — Opening the hood: from photons to tokens 17:11 — Extracting more value from the same chip 19:26 — Frontier inference and distributed training, explained 23:26 — What actually drives compute cost 25:21 — Lambda's chip stack and the NVIDIA relationship 26:17 — A multi-silicon world? CUDA, CUDNN, and NVIDIA's real moat 28:59 — Networking, storage, and the one-click cluster 34:46 — Renting vs. owning, and full vertical integration 36:24 — How global is Lambda? Does location still matter? 38:44 — The financing stack: off-take agreements, SPVs, and credit 41:16 — Why a 2023 GPU leases for more today 42:36 — A futures market for compute? 43:54 — Origin story: facial recognition, Perceptio, and Apple 47:03 — The Lambda hat and Dream Scope 48:59 — The $60K bet that became a cloud business 52:00 — Holding the team together through the hard times 54:30 — Bringing on a new CEO; Stephen as CTO 57:33 — Matching xAI on high-velocity deployment 59:29 — "AI won't write software — it will become the software" 01:01:30 — Neural software vs. vibe coding 01:04:25 — Do agents change the compute layer 01:06:14 — Self-assembling software inside Lambda 01:08:18 — Gigawatt-scale AI factories 01:08:57 — One person, one GPU 01:12:04 — Hot takes: overrated and underrated in AI

Matt Turck

70,916 views • 28 days ago

It's brain versus AI time. Here's our full episode with Sebastian Seung. He's started a new company to reverse engineer the brain and master the intelligence created by nature with the hopes of informing the intelligence we've created with machines. Sebastian has done pioneering AI and neuroscience work and is now blending the two to make a digital brain. Mostly, he's fun to listen to. The Core Memory podcast is made by possible by the intelligent finance powerhouse Brex and the people who back the biggest ideas @e1ventures. Timestamps 00:00:00 - Intro 00:01:42 - Who is Sebastian Seung? 00:03:23 - What is a Connectome? 00:07:08 - The Connectome Winter 00:09:08 - Diamond Knives & AI 00:12:10 - When Will We Map the Human Brain? 00:18:09 - The "Brain Crystal" & Visual Neurons 00:26:31 - New Startup 00:33:55 - Simulating a Mind in Software 00:37:40 - Biology vs. Nvidia 00:44:04 - Is AI a Religious Quest? 00:50:05 - Mind Uploading & Transhumanism 00:59:26 - Brain-Computer Interfaces 01:06:41 - From Physics to Neuroscience 01:12:22 - Will Uploads Be Conscious? 01:19:37 - Neuromorphic Chips & Future Robotics
1:28:39

It's brain versus AI time. Here's our full episode with Sebastian Seung. He's started a new company to reverse engineer the brain and master the intelligence created by nature with the hopes of informing the intelligence we've created with machines. Sebastian has done pioneering AI and neuroscience work and is now blending the two to make a digital brain. Mostly, he's fun to listen to. The Core Memory podcast is made by possible by the intelligent finance powerhouse Brex and the people who back the biggest ideas @e1ventures. Timestamps 00:00:00 - Intro 00:01:42 - Who is Sebastian Seung? 00:03:23 - What is a Connectome? 00:07:08 - The Connectome Winter 00:09:08 - Diamond Knives & AI 00:12:10 - When Will We Map the Human Brain? 00:18:09 - The "Brain Crystal" & Visual Neurons 00:26:31 - New Startup 00:33:55 - Simulating a Mind in Software 00:37:40 - Biology vs. Nvidia 00:44:04 - Is AI a Religious Quest? 00:50:05 - Mind Uploading & Transhumanism 00:59:26 - Brain-Computer Interfaces 01:06:41 - From Physics to Neuroscience 01:12:22 - Will Uploads Be Conscious? 01:19:37 - Neuromorphic Chips & Future Robotics

Ashlee Vance

38,233 views • 7 months ago