Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

New blackboard lecture w Reiner Pope How do chips actually work – starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do. 0:00:00 – Building a multiply-accumulate from logic gates 0:16:20 – Muxes and the cost... show more

Dwarkesh Patel

242,187 subscribers

932,625 views • 1 month ago •via X (Twitter)

Arts Education Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

The Jensen Huang episode. 0:00:00 – Is Nvidia’s biggest moat its grip on scarce supply chains? 0:16:25 – Will TPUs break Nvidia’s hold on AI compute? 0:41:06 – Why doesn’t Nvidia become a hyperscaler? 0:57:36 – Should we be selling AI chips to China? 1:35:06 – Why doesn’t Nvidia make multiple different chip architectures? Look up Dwarkesh Podcast on YouTube, Apple Podcasts, Spotify, etc. Enjoy!

The Jensen Huang episode. 0:00:00 – Is Nvidia’s biggest moat its grip on scarce supply chains? 0:16:25 – Will TPUs break Nvidia’s hold on AI compute? 0:41:06 – Why doesn’t Nvidia become a hyperscaler? 0:57:36 – Should we be selling AI chips to China? 1:35:06 – Why doesn’t Nvidia make multiple different chip architectures? Look up Dwarkesh Podcast on YouTube, Apple Podcasts, Spotify, etc. Enjoy!

Dwarkesh Patel

6,213,623 views • 3 months ago

The Dario Amodei interview. 0:00:00 - What exactly are we scaling? 0:12:36 - Is diffusion cope? 0:29:42 - Is continual learning necessary? 0:46:20 - If AGI is imminent, why not buy more compute? 0:58:49 - How will AI labs actually make profit? 1:31:19 - Will regulations destroy the boons of AGI? 1:47:41 - Why can’t China and America both have a country of geniuses in a datacenter? Look up Dwarkesh Podcast on Youtube, Spotify, Apple Podcasts, etc.

The Dario Amodei interview. 0:00:00 - What exactly are we scaling? 0:12:36 - Is diffusion cope? 0:29:42 - Is continual learning necessary? 0:46:20 - If AGI is imminent, why not buy more compute? 0:58:49 - How will AI labs actually make profit? 1:31:19 - Will regulations destroy the boons of AGI? 1:47:41 - Why can’t China and America both have a country of geniuses in a datacenter? Look up Dwarkesh Podcast on Youtube, Spotify, Apple Podcasts, etc.

Dwarkesh Patel

2,056,457 views • 5 months ago

The Ilya Sutskever episode 0:00:00 – Explaining model jaggedness 0:09:39 - Emotions and value functions 0:18:49 – What are we scaling? 0:25:13 – Why humans generalize better than models 0:35:45 – Straight-shotting superintelligence 0:46:47 – SSI’s model will learn from deployment 0:55:07 – Alignment 1:18:13 – “We are squarely an age of research company” 1:29:23 – Self-play and multi-agent 1:32:42 – Research taste Look up Dwarkesh Podcast on YouTube, Apple Podcasts, or Spotify. Enjoy!

The Ilya Sutskever episode 0:00:00 – Explaining model jaggedness 0:09:39 - Emotions and value functions 0:18:49 – What are we scaling? 0:25:13 – Why humans generalize better than models 0:35:45 – Straight-shotting superintelligence 0:46:47 – SSI’s model will learn from deployment 0:55:07 – Alignment 1:18:13 – “We are squarely an age of research company” 1:29:23 – Self-play and multi-agent 1:32:42 – Research taste Look up Dwarkesh Podcast on YouTube, Apple Podcasts, or Spotify. Enjoy!

Dwarkesh Patel

4,126,193 views • 7 months ago

NEW EPISODE: “The first step of converting fear into a habitable place is ask yourself, what am I afraid of and why am I afraid of this? It's not about banishing the fear–it's about asking yourself, what is it trying to teach me? What am I learning from this?” -Louis C.K. 0:00:00 Louis C.K. 0:00:14 How Jokes Are Found 0:07:25 Small Rooms vs. Big Crowds 0:14:11 Life on the Road & Taking Care of Yourself 0:21:30 Dumb Jokes, Dark Jokes & Gut Instinct 0:40:40 Rhythm, Timing & the Craft of Comedy 0:49:41 Timeless vs. Topical Material 1:00:14 Saying the Unsayable on Stage 1:16:22 What Actually Makes Something Funny 1:34:48 Structuring a Set & Staying Awake 1:43:02 Boston, New York & Comedy Lineage

NEW EPISODE: “The first step of converting fear into a habitable place is ask yourself, what am I afraid of and why am I afraid of this? It's not about banishing the fear–it's about asking yourself, what is it trying to teach me? What am I learning from this?” -Louis C.K. 0:00:00 Louis C.K. 0:00:14 How Jokes Are Found 0:07:25 Small Rooms vs. Big Crowds 0:14:11 Life on the Road & Taking Care of Yourself 0:21:30 Dumb Jokes, Dark Jokes & Gut Instinct 0:40:40 Rhythm, Timing & the Craft of Comedy 0:49:41 Timeless vs. Topical Material 1:00:14 Saying the Unsayable on Stage 1:16:22 What Actually Makes Something Funny 1:34:48 Structuring a Set & Staying Awake 1:43:02 Boston, New York & Comedy Lineage

Tetragrammaton

100,663 views • 13 days ago

The Andrej Karpathy interview 0:00:00 – AGI is still a decade away 0:30:33 – LLM cognitive deficits 0:40:53 – RL is terrible 0:50:26 – How do humans learn? 1:07:13 – AGI will blend into 2% GDP growth 1:18:24 – ASI 1:33:38 – Evolution of intelligence & culture 1:43:43 - Why self driving took so long 1:57:08 - Future of education Look up Dwarkesh Podcast on YouTube, Apple Podcasts, Spotify, etc. Enjoy!

The Andrej Karpathy interview 0:00:00 – AGI is still a decade away 0:30:33 – LLM cognitive deficits 0:40:53 – RL is terrible 0:50:26 – How do humans learn? 1:07:13 – AGI will blend into 2% GDP growth 1:18:24 – ASI 1:33:38 – Evolution of intelligence & culture 1:43:43 - Why self driving took so long 1:57:08 - Future of education Look up Dwarkesh Podcast on YouTube, Apple Podcasts, Spotify, etc. Enjoy!

Dwarkesh Patel

10,754,672 views • 9 months ago

New episode w Adam Marblestone on what the brain's secret sauce is: how do we learn so much from so little? Also, the answer to Ilya’s question: how does the genome encode desires for high level concepts that are only seen during lifetime? Turns out, they’re deeply connected questions. Timestamps 0:00:00 – The brain’s secret sauce is the reward functions, not the architecture 0:22:20 – What the genome actually encodes 0:42:42 – What kind of RL is the brain doing? 0:50:31 – Is biological hardware a limitation or an advantage? 1:03:59 – Why we need to map the human brain 1:23:28 – What value will automating math have? 1:38:18 – Architecture of the brain Look up Dwarkesh Podcast on YouTube, Apple Podcasts, Spotify, etc. Enjoy!

New episode w Adam Marblestone on what the brain's secret sauce is: how do we learn so much from so little? Also, the answer to Ilya’s question: how does the genome encode desires for high level concepts that are only seen during lifetime? Turns out, they’re deeply connected questions. Timestamps 0:00:00 – The brain’s secret sauce is the reward functions, not the architecture 0:22:20 – What the genome actually encodes 0:42:42 – What kind of RL is the brain doing? 0:50:31 – Is biological hardware a limitation or an advantage? 1:03:59 – Why we need to map the human brain 1:23:28 – What value will automating math have? 1:38:18 – Architecture of the brain Look up Dwarkesh Podcast on YouTube, Apple Podcasts, Spotify, etc. Enjoy!

Dwarkesh Patel

746,125 views • 6 months ago

New episode w Lewis Bollard - a deep dive on the surprising economics of the meat industry. 0:00:00 – The astonishing efficiency of factory farming 0:07:18 – It was a mistake making this about diet 0:09:54 – Tech that’s sparing 100s of millions of animals/year 0:16:16 – Brainless chickens and higher welfare breeds 0:28:21 – $1 can prevent 10 years of animal suffering 0:37:26 – The situation in China and the developing world 0:41:41 – How the meat lobby got a lock on Congress 0:53:23 – Business structure of the meat industry 0:57:42 – Corporate campaigns are underrated Available on YouTube, Apple Podcasts, Spotify, etc (look up Dwarkesh Podcast).

New episode w Lewis Bollard - a deep dive on the surprising economics of the meat industry. 0:00:00 – The astonishing efficiency of factory farming 0:07:18 – It was a mistake making this about diet 0:09:54 – Tech that’s sparing 100s of millions of animals/year 0:16:16 – Brainless chickens and higher welfare breeds 0:28:21 – $1 can prevent 10 years of animal suffering 0:37:26 – The situation in China and the developing world 0:41:41 – How the meat lobby got a lock on Congress 0:53:23 – Business structure of the meat industry 0:57:42 – Corporate campaigns are underrated Available on YouTube, Apple Podcasts, Spotify, etc (look up Dwarkesh Podcast).

Dwarkesh Patel

240,050 views • 11 months ago

How Russia derailed China’s rise for over a century: new lecture and Q&A w Sarah Paine. IMO the Chinese Civil War is 1 of the top 3 most important events of the 20th century. To understand why it transpired as it did, you need to understand Stalin’s role in the whole thing. Look up Dwarkesh Podcast on YouTube, Apple Podcasts, or Spotify to catch it there. Enjoy! 0:00:00 – How Russia took advantage of China's weakness 0:23:46 – After Stalin, China's rise 0:34:39 – Russian imperialism 0:46:10 – China and Russia's existential problems 1:05:42 – Q&A: Sino-Soviet split 1:23:29 – Stalin's lessons from WW2

How Russia derailed China’s rise for over a century: new lecture and Q&A w Sarah Paine. IMO the Chinese Civil War is 1 of the top 3 most important events of the 20th century. To understand why it transpired as it did, you need to understand Stalin’s role in the whole thing. Look up Dwarkesh Podcast on YouTube, Apple Podcasts, or Spotify to catch it there. Enjoy! 0:00:00 – How Russia took advantage of China's weakness 0:23:46 – After Stalin, China's rise 0:34:39 – Russian imperialism 0:46:10 – China and Russia's existential problems 1:05:42 – Q&A: Sino-Soviet split 1:23:29 – Stalin's lessons from WW2

Dwarkesh Patel

57,358 views • 8 months ago

Our new podcast on evals, with Max Niederman, Ege Erdil, and Stephen Yang. 0:00:00 – What's an eval, and how's it different from an RL environment? 0:19:33 – Why are models bad at building an emulator when the task is fully verifiable? 0:42:00 – How does training on bad data teach models to write terrible code? 1:04:00 – Why is continual learning still so bad? 2:25:24 – Why haven't software engineers been replaced when coding is basically solved? Listen to the Mechanize Podcast on YouTube, Spotify, etc. Enjoy!

Our new podcast on evals, with Max Niederman, Ege Erdil, and Stephen Yang. 0:00:00 – What's an eval, and how's it different from an RL environment? 0:19:33 – Why are models bad at building an emulator when the task is fully verifiable? 0:42:00 – How does training on bad data teach models to write terrible code? 1:04:00 – Why is continual learning still so bad? 2:25:24 – Why haven't software engineers been replaced when coding is basically solved? Listen to the Mechanize Podcast on YouTube, Spotify, etc. Enjoy!

Mechanize

38,541 views • 28 days ago

Jorge Masvidal’s Prediction For UFC 302 / Fighting McGregor?! | TSS | EP.287 0:00:00 - Intro 0:00:20 - Montana Trip Recap 0:01:13 - Dumb & Dumber To 0:03:45 - Fast Food Challenge 0:07:33 - Schmitty Goes On A Date 0:08:50 - Suga & Tim’s Chickens 0:10:00 - Tammy Might Be Preg.. 0:11:25 - Does Dustin Guillotine Islam? 0:15:04 - Strickland Vs Coasta Prediction 0:17:46 - Tim’s New Tesla 0:18:55 - McGregor Explains Comments Towards Suga 0:21:10 - Suga’s Warehouse Upgrades 0:23:53 - Suga Announces Next Fight 0:26:08 - Schmitty Vs Merab Beef 0:27:27 - Private Jet Scare Story 0:28:40 - Stoicism Levels 0:32:12 - Upcoming Business Trips 0:34:16 - Types Of Humor 0:36:50 - Do You Care What People Think About You? 0:38:55 - Ozempic For Weight Loss 0:40:10 - First Neuralink Patient 0:41:50 - Suga Meeting Masvidal For The First Time 0:42:42 - Jobin TV Show Update 0:44:55 - Chito Vera Vs Figgy Prediction 0:45:55 - Ian Gary Vs MVP Prediction 0:48:25 - Everyone Sings For Each Other 0:49:00 - Suga Visiting His Home He Grew Up In 0:52:32 - Tim Debates W/ His Mom 0:54:45 - Schmitty Has A Flare Up 0:55:44 - Henry Cejudo Turning Down Fights 0:56:55 - Movie Night 0:58:24 - Welcome Jorge Masvidal 0:59:00 - Boxing Camp For Nate Diaz 1:01:07 - Difference From Prepping For An MMA Fight 1:02:28 - How Masvidal’s Diet Has Changed With Age 1:03:50 - The Biggest Fear Going Into A Fight 1:05:12 - The Risk Of Going For A Flying Knee Against Ben Askren 1:06:55 - Downtown During Camp 1:08:10 - Studying Boxing Film 1:09:50 - Fighting Jake Paul In MMA 1:10:55 - Jorge’s Prediction For Dustin Vs Islam 1:12:55 - Masvidal Vs McGregor? 1:13:30 - Chael Vs Anderson Boxing Prediction 1:14:35 - Miami Lifestyle 1:15:33 - Masvidal’s Biggest Regret In His Fight Career 1:19:20 - Masvidal Gives Suga Advice 1:22:30 - Rumors On Merab Fight 1:24:00 - Chandler Vs McGregor Prediction

Jorge Masvidal’s Prediction For UFC 302 / Fighting McGregor?! | TSS | EP.287 0:00:00 - Intro 0:00:20 - Montana Trip Recap 0:01:13 - Dumb & Dumber To 0:03:45 - Fast Food Challenge 0:07:33 - Schmitty Goes On A Date 0:08:50 - Suga & Tim’s Chickens 0:10:00 - Tammy Might Be Preg.. 0:11:25 - Does Dustin Guillotine Islam? 0:15:04 - Strickland Vs Coasta Prediction 0:17:46 - Tim’s New Tesla 0:18:55 - McGregor Explains Comments Towards Suga 0:21:10 - Suga’s Warehouse Upgrades 0:23:53 - Suga Announces Next Fight 0:26:08 - Schmitty Vs Merab Beef 0:27:27 - Private Jet Scare Story 0:28:40 - Stoicism Levels 0:32:12 - Upcoming Business Trips 0:34:16 - Types Of Humor 0:36:50 - Do You Care What People Think About You? 0:38:55 - Ozempic For Weight Loss 0:40:10 - First Neuralink Patient 0:41:50 - Suga Meeting Masvidal For The First Time 0:42:42 - Jobin TV Show Update 0:44:55 - Chito Vera Vs Figgy Prediction 0:45:55 - Ian Gary Vs MVP Prediction 0:48:25 - Everyone Sings For Each Other 0:49:00 - Suga Visiting His Home He Grew Up In 0:52:32 - Tim Debates W/ His Mom 0:54:45 - Schmitty Has A Flare Up 0:55:44 - Henry Cejudo Turning Down Fights 0:56:55 - Movie Night 0:58:24 - Welcome Jorge Masvidal 0:59:00 - Boxing Camp For Nate Diaz 1:01:07 - Difference From Prepping For An MMA Fight 1:02:28 - How Masvidal’s Diet Has Changed With Age 1:03:50 - The Biggest Fear Going Into A Fight 1:05:12 - The Risk Of Going For A Flying Knee Against Ben Askren 1:06:55 - Downtown During Camp 1:08:10 - Studying Boxing Film 1:09:50 - Fighting Jake Paul In MMA 1:10:55 - Jorge’s Prediction For Dustin Vs Islam 1:12:55 - Masvidal Vs McGregor? 1:13:30 - Chael Vs Anderson Boxing Prediction 1:14:35 - Miami Lifestyle 1:15:33 - Masvidal’s Biggest Regret In His Fight Career 1:19:20 - Masvidal Gives Suga Advice 1:22:30 - Rumors On Merab Fight 1:24:00 - Chandler Vs McGregor Prediction

Sean O'Malley

118,091 views • 2 years ago

.Satya Nadella gave me and Dylan Patel an exclusive tour of Fairwater 2, the most powerful AI datacenter in the world. We then chatted through Satya's vision for Microsoft in a world with AGI. 0:00:00 - Fairwater 2 0:04:15 - Business models for AGI 0:13:42 - Copilot 0:20:56 - Whose margins will expand most? 0:37:12 - MAI 0:48:42 - The hyperscale business 1:03:39 - In-house chip & OpenAI partnership 1:10:30 - The CAPEX explosion 1:16:01 - Will the world trust US companies to lead AI? Look up Dwarkesh Podcast on Youtube, Apple Podcasts or Spotify to tune in.

.Satya Nadella gave me and Dylan Patel an exclusive tour of Fairwater 2, the most powerful AI datacenter in the world. We then chatted through Satya's vision for Microsoft in a world with AGI. 0:00:00 - Fairwater 2 0:04:15 - Business models for AGI 0:13:42 - Copilot 0:20:56 - Whose margins will expand most? 0:37:12 - MAI 0:48:42 - The hyperscale business 1:03:39 - In-house chip & OpenAI partnership 1:10:30 - The CAPEX explosion 1:16:01 - Will the world trust US companies to lead AI? Look up Dwarkesh Podcast on Youtube, Apple Podcasts or Spotify to tune in.

Dwarkesh Patel

1,895,028 views • 8 months ago

A Quest to Formalize Intelligence ft. ludwig 0:00:00 - INTRO 0:01:06 - What is he most excited about these days 0:03:31 - All arcs are isomorphic, Dropping out in 7th grade 0:09:31 - Does skipping formal education make you naturally broader? 0:10:11 - Growing up on 4chan, Mathematics is humbling 0:14:52 - Mentors, First programming job, Internet microcultures 0:21:47 - Culture in SF, Why he won't start a company 0:25:21 - Does AI make self-teaching easier or trap you in infinite explanations? 0:29:49 - The 20% doing 80% inside a company 0:33:24 - Company culture and Building subcultures 0:35:56 - Define understanding, Compression equals Prediction 0:45:44 - Understanding vs knowing, Michael Levin, Cognitive light cones 0:54:39 - Starting a Research lab, Sheaf theory and Grothendieck 0:58:43 - Mapping Levin's biology onto proof space with MCTS and Lean4 1:02:49 - Active inference, Distributed systems and Morphogenesis 1:12:28 - Signal vs noise on X, The unit distance proof, Flanderization 1:17:39 - Community Questions 1:21:39 - Shape rotator vs wordcel, Tenstorrent, How to hire great engineers 1:28:01 - Building mental space to do things that will be left to do 1:33:42 - Advice to a 20-year-old. Be a capable thinker, you have 20k days left

A Quest to Formalize Intelligence ft. ludwig 0:00:00 - INTRO 0:01:06 - What is he most excited about these days 0:03:31 - All arcs are isomorphic, Dropping out in 7th grade 0:09:31 - Does skipping formal education make you naturally broader? 0:10:11 - Growing up on 4chan, Mathematics is humbling 0:14:52 - Mentors, First programming job, Internet microcultures 0:21:47 - Culture in SF, Why he won't start a company 0:25:21 - Does AI make self-teaching easier or trap you in infinite explanations? 0:29:49 - The 20% doing 80% inside a company 0:33:24 - Company culture and Building subcultures 0:35:56 - Define understanding, Compression equals Prediction 0:45:44 - Understanding vs knowing, Michael Levin, Cognitive light cones 0:54:39 - Starting a Research lab, Sheaf theory and Grothendieck 0:58:43 - Mapping Levin's biology onto proof space with MCTS and Lean4 1:02:49 - Active inference, Distributed systems and Morphogenesis 1:12:28 - Signal vs noise on X, The unit distance proof, Flanderization 1:17:39 - Community Questions 1:21:39 - Shape rotator vs wordcel, Tenstorrent, How to hire great engineers 1:28:01 - Building mental space to do things that will be left to do 1:33:42 - Advice to a 20-year-old. Be a capable thinker, you have 20k days left

himanshu

39,650 views • 1 month ago

We are excited to share that Logan Kilpatrick joined us on The Bench to discuss Google's new Gemini 3.5 Flash: why it's deliberately more persistent and capable than previous Flash models, how it hit #1 on our FinanceAgent Benchmark taking 82 steps where competitors stopped at 13, and what justifies the price increase. We also get into why AI benchmarks need a paradigm shift, the trade-off of building everything vs staying focused, the Pope, and why Omni might kill the Subway Surfers content era. 0:11:00 – Flash is being rebased for the agent era, not just a cheaper model anymore 0:14:03 – Persistence by design: 82 tool calls vs competitors' 13 0:17:52 – Why pricing went up and how Google thinks about value per token 0:22:55 – Coding performance: from 20th to 10th place in one generation 0:28:28 – Why benchmarks have historically been misleading and what the new era of evaluation looks like 0:29:28 Logan on why Google has the best researchers in the world 0:36:16 – The cost of being Google 0:39:07 – The Pope’s encyclical on AI and whether most people see frontier intelligence as a good thing 0:51:12 – Why Omni is the thing that recently clicked

We are excited to share that Logan Kilpatrick joined us on The Bench to discuss Google's new Gemini 3.5 Flash: why it's deliberately more persistent and capable than previous Flash models, how it hit #1 on our FinanceAgent Benchmark taking 82 steps where competitors stopped at 13, and what justifies the price increase. We also get into why AI benchmarks need a paradigm shift, the trade-off of building everything vs staying focused, the Pope, and why Omni might kill the Subway Surfers content era. 0:11:00 – Flash is being rebased for the agent era, not just a cheaper model anymore 0:14:03 – Persistence by design: 82 tool calls vs competitors' 13 0:17:52 – Why pricing went up and how Google thinks about value per token 0:22:55 – Coding performance: from 20th to 10th place in one generation 0:28:28 – Why benchmarks have historically been misleading and what the new era of evaluation looks like 0:29:28 Logan on why Google has the best researchers in the world 0:36:16 – The cost of being Google 0:39:07 – The Pope’s encyclical on AI and whether most people see frontier intelligence as a good thing 0:51:12 – Why Omni is the thing that recently clicked

Vals AI

16,486 views • 1 month ago

Episode w Dylan Patel and Asianometry out! A bonanza on how the semiconductor industry actually works. & what Xi could do if he becomes scaling pilled, how we can train models with 10,000x GPT-4's compute by end of decade, & much more. This one is very fun. Enjoy! Links below. 0:00:00 – Xi’s path to AGI 0:05:05 – Liang Mong Song 0:09:10 – How semiconductors get better 0:12:01 – China can centralize compute 0:19:35 – Export controls & sanctions 0:33:45 – Huawei’s intense culture 0:39:45 – Why the semiconductor industry is so stratified 0:41:50 – N2 should not exist 0:46:47 – Taiwan invasion hypothetical 0:50:15 – Mind-boggling complexity of semiconductors 1:00:06 – Chip architecture design 1:05:29 – Architectures lead to different AI models? China vs. US 1:11:06 – Being head of compute at an AI lab 1:17:17 – Scaling costs and power demand 1:37:59 – Are we financing an AI bubble?

Episode w Dylan Patel and Asianometry out! A bonanza on how the semiconductor industry actually works. & what Xi could do if he becomes scaling pilled, how we can train models with 10,000x GPT-4's compute by end of decade, & much more. This one is very fun. Enjoy! Links below. 0:00:00 – Xi’s path to AGI 0:05:05 – Liang Mong Song 0:09:10 – How semiconductors get better 0:12:01 – China can centralize compute 0:19:35 – Export controls & sanctions 0:33:45 – Huawei’s intense culture 0:39:45 – Why the semiconductor industry is so stratified 0:41:50 – N2 should not exist 0:46:47 – Taiwan invasion hypothetical 0:50:15 – Mind-boggling complexity of semiconductors 1:00:06 – Chip architecture design 1:05:29 – Architectures lead to different AI models? China vs. US 1:11:06 – Being head of compute at an AI lab 1:17:17 – Scaling costs and power demand 1:37:59 – Are we financing an AI bubble?

Dwarkesh Patel

260,187 views • 1 year ago

Our episode with aerospace investor and York Space Systems Chairman Charles "Chuck" Beames is now live! Chuck joins us to discuss: 🚀 The Dawn of the Second Space Race 🔐 Cybersecurity and Satellites 🎯 Wargaming with AIs and Humans ➿ The Complexity of Space Procurement ♾ and MUCH more! Timestamps: 0:00:00 Intro 0:01:01 Guest introduction 0:02:15 The Second Space Race 0:07:52 The Zero Trust Approach and 'Secure By Design' 0:16:01 GPS as a Military Technology 0:20:22 The Space Force is dealing with Multiple Complex Adaptive Systems 0:28:21 New Paradigms in Satellite Design 0:31:20 What VCs and Bankers Need to know about the Space Community 0:40:17 Wargaming with AIs and Humans 0:51:11 Laser Spoofing and Adversarial AIs 1:00:10 Establishing Supply Chain Sovereignty 1:05:13 Open Organizational Cultures 1:10:59 The importance of working with your hands 1:21:22 The Job of an Executive Chairman 1:26:41 Chuck as World Emperor

Our episode with aerospace investor and York Space Systems Chairman Charles "Chuck" Beames is now live! Chuck joins us to discuss: 🚀 The Dawn of the Second Space Race 🔐 Cybersecurity and Satellites 🎯 Wargaming with AIs and Humans ➿ The Complexity of Space Procurement ♾ and MUCH more! Timestamps: 0:00:00 Intro 0:01:01 Guest introduction 0:02:15 The Second Space Race 0:07:52 The Zero Trust Approach and 'Secure By Design' 0:16:01 GPS as a Military Technology 0:20:22 The Space Force is dealing with Multiple Complex Adaptive Systems 0:28:21 New Paradigms in Satellite Design 0:31:20 What VCs and Bankers Need to know about the Space Community 0:40:17 Wargaming with AIs and Humans 0:51:11 Laser Spoofing and Adversarial AIs 1:00:10 Establishing Supply Chain Sovereignty 1:05:13 Open Organizational Cultures 1:10:59 The importance of working with your hands 1:21:22 The Job of an Executive Chairman 1:26:41 Chuck as World Emperor

Infinite Loops 🎙

13,374 views • 1 year ago

.Dylan Patel gives a deep dive on the 3 big bottlenecks to scaling AI compute: logic, memory, and power. And walks through the economics of labs, hyperscalers, foundries, and fab equipment manufacturers. Learned a ton about every single level of the stack. 0:00:00 – Why an H100 is worth more today than 3 years ago 0:24:52 – Nvidia secured TSMC allocation early; Google is getting squeezed 0:34:34 – ASML will be the #1 constraint for AI compute scaling by 2030 0:56:06 – Can’t we just use TSMC’s older fabs? 1:05:56 – When will China outscale the West in semis? 1:16:20 – The enormous incoming memory crunch 1:42:53 – Scaling power in the US will not be a problem 1:55:03 – Space GPUs aren't happening this decade 2:14:26 – Why aren’t more hedge funds making the AGI trade? 2:18:49 – Will TSMC kick Apple out from N2? 2:24:35 – Robots and Taiwan risk Look up Dwarkesh Podcast on YouTube, Apple Podcasts, or Spotify. Enjoy!

.Dylan Patel gives a deep dive on the 3 big bottlenecks to scaling AI compute: logic, memory, and power. And walks through the economics of labs, hyperscalers, foundries, and fab equipment manufacturers. Learned a ton about every single level of the stack. 0:00:00 – Why an H100 is worth more today than 3 years ago 0:24:52 – Nvidia secured TSMC allocation early; Google is getting squeezed 0:34:34 – ASML will be the #1 constraint for AI compute scaling by 2030 0:56:06 – Can’t we just use TSMC’s older fabs? 1:05:56 – When will China outscale the West in semis? 1:16:20 – The enormous incoming memory crunch 1:42:53 – Scaling power in the US will not be a problem 1:55:03 – Space GPUs aren't happening this decade 2:14:26 – Why aren’t more hedge funds making the AGI trade? 2:18:49 – Will TSMC kick Apple out from N2? 2:24:35 – Robots and Taiwan risk Look up Dwarkesh Podcast on YouTube, Apple Podcasts, or Spotify. Enjoy!

Dwarkesh Patel

489,541 views • 4 months ago

GPU by hand ✍️ I drew 42 frames to show how a GPU speeds up an array operation of 8 elements in parallel over 4 threads in 2 clock cycles. Below is an overview: CPU • It has one core. • Its global memory has 120 locations (0-119). • To use the GPU, it needs to copy data from the global memory to the GPU. • After GPU is done, it will copy the results back. GPU • It has four cores to run four threads (0-3). • It has a register file of 28 locations (0-27) • This register file has four banks (0-3). • All threads share the same register file. • But they must read/write using the four banks. • Each bank allows 2 reads (Read 0, Read 1) and 1 write in a single clock cycle.

GPU by hand ✍️ I drew 42 frames to show how a GPU speeds up an array operation of 8 elements in parallel over 4 threads in 2 clock cycles. Below is an overview: CPU • It has one core. • Its global memory has 120 locations (0-119). • To use the GPU, it needs to copy data from the global memory to the GPU. • After GPU is done, it will copy the results back. GPU • It has four cores to run four threads (0-3). • It has a register file of 28 locations (0-27) • This register file has four banks (0-3). • All threads share the same register file. • But they must read/write using the four banks. • Each bank allows 2 reads (Read 0, Read 1) and 1 write in a single clock cycle.

Tom Yeh

101,788 views • 1 year ago

The Jeff Dean & Noam Shazeer episode. We talk about 25 years at Google, from PageRank to MapReduce to the Transformer to MoEs to AlphaChip – and soon to ASI. My favorite part was Jeff's vision for AGI as one giant MoE that is grown in bits and pieces over time like a forest, rather than trained all at once. Specialization, distillation, inference time scaling all emerge organically rather than by design. Noam bites every bullet: 100x world GDP soon; let’s get a million automated researchers running in the Google datacenter; living to see the year 3000. Links below. Enjoy! Timestamps 0:00:00 - Intro 0:03:29 - Joining Google in 1999 0:06:20 - Future of Moore's Law 0:11:04 - Future TPUs 0:13:56 - Jeff’s undergrad thesis: parallel backprop 0:15:54 - LLMs in 2007 0:25:09 - “Holy shit” moments 0:27:28 - AI fulfills Google’s original mission 0:32:00 - Doing Search in-context 0:36:12 - The internal coding model 0:37:29 - What will 2027 models do? 0:43:20 - A new architecture every day? 0:49:10 - Automated chips and intelligence explosion 0:53:07 - Future of inference scaling 1:02:38 - Already doing multi-datacenter runs 1:08:15 - Debugging at scale 1:12:41 - Fast takeoff and superalignment 1:20:51 - A million evil Jeff Deans 1:24:22 - Fun times at Google 1:27:51 - World compute demand in 2030 1:34:37 - Getting back to modularity 1:44:48 - Keeping a giga-MoE in-memory 1:49:35 - All of Google in one model 1:57:59 - What’s missing from distillation 2:03:10 - Open research, pros and cons 2:09:58 - Going the distance

The Jeff Dean & Noam Shazeer episode. We talk about 25 years at Google, from PageRank to MapReduce to the Transformer to MoEs to AlphaChip – and soon to ASI. My favorite part was Jeff's vision for AGI as one giant MoE that is grown in bits and pieces over time like a forest, rather than trained all at once. Specialization, distillation, inference time scaling all emerge organically rather than by design. Noam bites every bullet: 100x world GDP soon; let’s get a million automated researchers running in the Google datacenter; living to see the year 3000. Links below. Enjoy! Timestamps 0:00:00 - Intro 0:03:29 - Joining Google in 1999 0:06:20 - Future of Moore's Law 0:11:04 - Future TPUs 0:13:56 - Jeff’s undergrad thesis: parallel backprop 0:15:54 - LLMs in 2007 0:25:09 - “Holy shit” moments 0:27:28 - AI fulfills Google’s original mission 0:32:00 - Doing Search in-context 0:36:12 - The internal coding model 0:37:29 - What will 2027 models do? 0:43:20 - A new architecture every day? 0:49:10 - Automated chips and intelligence explosion 0:53:07 - Future of inference scaling 1:02:38 - Already doing multi-datacenter runs 1:08:15 - Debugging at scale 1:12:41 - Fast takeoff and superalignment 1:20:51 - A million evil Jeff Deans 1:24:22 - Fun times at Google 1:27:51 - World compute demand in 2030 1:34:37 - Getting back to modularity 1:44:48 - Keeping a giga-MoE in-memory 1:49:35 - All of Google in one model 1:57:59 - What’s missing from distillation 2:03:10 - Open research, pros and cons 2:09:58 - Going the distance

Dwarkesh Patel

544,788 views • 1 year ago

New episode with Francis Fukuyama (Francis Fukuyama). While The End of History and the Last Man is arguably the most influential work in political science of the past half-century, I wanted to focus on one of Frank's lesser-known books: Our Posthuman Future. In that book, he considers biotech and its potential to usher in a new, post-human history. I check in to see how his views have evolved since it was published (2002), then ask him about AI and its potential to restart "History". The goal was to discuss topics that are important and on which Frank has spoken little until now. Frank also gives me a preview of his forthcoming book, and shares some ideas on delegation and state capacity. (0:00:00) - Introduction. (0:00:41) - Are new biotechnologies restarting History? (0:03:52) - Why Fukuyama thinks life extension is a bad idea. (0:06:47) - Which biotechnology is most likely to drive transhumanism? (0:07:34) - Political orders are downstream of technology. (0:10:31) - Would Fukuyama have denied political rights to Denisovans or Neanderthals? (0:13:56) - Transhumanism and longevity's unintended consequences. (0:18:55) - Assortative mating vs. gene editing. (0:20:00) - How should we adjudicate disagreements between different sets of natural rights? (0:24:03) - How is Fukuyama thinking about AGI and ASI? (0:27:31) - Granting AIs political rights? (0:37:23) - Best political order for a society of non-thymotic AIs? (0:40:44) - Could an ASI design a better political order than liberal democracy for today's world? (0:44:08) - Why Asian cultures are more permissive of biotechnology. (0:47:04) - Do LLMs represent a victory of American soft power in China? (0:49:17) - If humanity is made economically redundant, where will megalothymia be channelled? (0:51:58) - Can Trump and Musk's relationship endure their mutual megalothymia? (0:53:23) - A future without work? Kojève's 1959 trip to Japan, & the early-modern landed gentry. (1:00:41) - Hegel vs. Kojève: wars after history? (1:02:26) - What more would have to happen for China to falsify the end of history thesis? (1:03:39) - What Fukuyama learned from having his thesis chronically misunderstood. (1:06:37) - Fukuyama's advice for public intellectuals who want sustainable careers. (1:08:52) - A sneak peek at Fukuyama's next book. (1:10:32) - Delegation in armies and bureaucracies. (1:15:48) - Why Fukuyama thinks DOGE was misguided. (1:17:53) - The relationship between talent, delegation and state capacity. Watch full episode on YouTube, Spotify, Apple Podcasts, etc. Transcript at my website jnwpod dot com. Enjoy!

New episode with Francis Fukuyama (Francis Fukuyama). While The End of History and the Last Man is arguably the most influential work in political science of the past half-century, I wanted to focus on one of Frank's lesser-known books: Our Posthuman Future. In that book, he considers biotech and its potential to usher in a new, post-human history. I check in to see how his views have evolved since it was published (2002), then ask him about AI and its potential to restart "History". The goal was to discuss topics that are important and on which Frank has spoken little until now. Frank also gives me a preview of his forthcoming book, and shares some ideas on delegation and state capacity. (0:00:00) - Introduction. (0:00:41) - Are new biotechnologies restarting History? (0:03:52) - Why Fukuyama thinks life extension is a bad idea. (0:06:47) - Which biotechnology is most likely to drive transhumanism? (0:07:34) - Political orders are downstream of technology. (0:10:31) - Would Fukuyama have denied political rights to Denisovans or Neanderthals? (0:13:56) - Transhumanism and longevity's unintended consequences. (0:18:55) - Assortative mating vs. gene editing. (0:20:00) - How should we adjudicate disagreements between different sets of natural rights? (0:24:03) - How is Fukuyama thinking about AGI and ASI? (0:27:31) - Granting AIs political rights? (0:37:23) - Best political order for a society of non-thymotic AIs? (0:40:44) - Could an ASI design a better political order than liberal democracy for today's world? (0:44:08) - Why Asian cultures are more permissive of biotechnology. (0:47:04) - Do LLMs represent a victory of American soft power in China? (0:49:17) - If humanity is made economically redundant, where will megalothymia be channelled? (0:51:58) - Can Trump and Musk's relationship endure their mutual megalothymia? (0:53:23) - A future without work? Kojève's 1959 trip to Japan, & the early-modern landed gentry. (1:00:41) - Hegel vs. Kojève: wars after history? (1:02:26) - What more would have to happen for China to falsify the end of history thesis? (1:03:39) - What Fukuyama learned from having his thesis chronically misunderstood. (1:06:37) - Fukuyama's advice for public intellectuals who want sustainable careers. (1:08:52) - A sneak peek at Fukuyama's next book. (1:10:32) - Delegation in armies and bureaucracies. (1:15:48) - Why Fukuyama thinks DOGE was misguided. (1:17:53) - The relationship between talent, delegation and state capacity. Watch full episode on YouTube, Spotify, Apple Podcasts, etc. Transcript at my website jnwpod dot com. Enjoy!

Joseph Noel Walker

15,483 views • 11 months ago

.george church is the godfather of modern synthetic biology. He thinks that the last few decades of progress have put us on the verge of the big payoffs: deaging, deextinction, biobots that combine the best of human & natural engineering, and (unfortunately) weaponized mirror life. 0:00:00 – Aging solved by 2050 0:07:37 – Finding the master switch for any trait 0:19:50 – Weaponized mirror life 0:30:40 – Why hasn’t sequencing/synthesis led to biotech revolution? 0:50:26 – Impact of AGI on biology research progress 1:00:35 – Biobots that use the best of biological and human engineering 1:05:09 – Odds of life in universe 1:09:57 – Is DNA the ultimate data storage? 1:13:55 – Curing rare diseases with genetic counseling 1:22:23 – NIH & NSF budget cuts 1:25:26 – How one lab spawned 100 biotech companies Watch full episode on YouTube, Spotify, Apple Podcasts, etc. Transcript at my website dwarkesh dot com. Enjoy!

.george church is the godfather of modern synthetic biology. He thinks that the last few decades of progress have put us on the verge of the big payoffs: deaging, deextinction, biobots that combine the best of human & natural engineering, and (unfortunately) weaponized mirror life. 0:00:00 – Aging solved by 2050 0:07:37 – Finding the master switch for any trait 0:19:50 – Weaponized mirror life 0:30:40 – Why hasn’t sequencing/synthesis led to biotech revolution? 0:50:26 – Impact of AGI on biology research progress 1:00:35 – Biobots that use the best of biological and human engineering 1:05:09 – Odds of life in universe 1:09:57 – Is DNA the ultimate data storage? 1:13:55 – Curing rare diseases with genetic counseling 1:22:23 – NIH & NSF budget cuts 1:25:26 – How one lab spawned 100 biotech companies Watch full episode on YouTube, Spotify, Apple Podcasts, etc. Transcript at my website dwarkesh dot com. Enjoy!

Dwarkesh Patel

85,882 views • 1 year ago