Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Did a very different format with Reiner Pope – a blackboard lecture where he walks through how frontier LLMs are trained and served. It's shocking how much you can deduce about what the labs are doing from a handful of equations, public API prices, and some chalk. It’s a...

dwarkesh_sp's profile picture

Dwarkesh Patel

240,465 subscribers

1,285,605 Aufrufe • vor 2 Monaten •via X (Twitter)

BildungNachrichten & PolitikWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

.Reiner Pope works out from first principles how much frontier models are overtrained relative to Chinchilla optimal. One of the cleverest deductions from the lecture, and the one I enjoyed the most.
7:35

.Reiner Pope works out from first principles how much frontier models are overtrained relative to Chinchilla optimal. One of the cleverest deductions from the lecture, and the one I enjoyed the most.

Dwarkesh Patel

54,976 Aufrufe • vor 2 Monaten

New blackboard lecture w Reiner Pope How do chips actually work – starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do. 0:00:00 – Building a multiply-accumulate from logic gates 0:16:20 – Muxes and the cost of data movement 0:25:59 – How systolic arrays work 0:39:00 – Clock cycles and pipeline registers 0:51:40 – FPGAs vs ASICs 1:03:14 – Cache vs scratchpad 1:07:16 – Why CPU cores are much bigger than GPU cores 1:11:49 – Brains vs chips 1:15:22 – A GPU is just a bunch of tiny TPUs Look up Dwarkesh Podcast on YouTube/Spotify/etc to watch. Enjoy!
1:20:19

New blackboard lecture w Reiner Pope How do chips actually work – starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do. 0:00:00 – Building a multiply-accumulate from logic gates 0:16:20 – Muxes and the cost of data movement 0:25:59 – How systolic arrays work 0:39:00 – Clock cycles and pipeline registers 0:51:40 – FPGAs vs ASICs 1:03:14 – Cache vs scratchpad 1:07:16 – Why CPU cores are much bigger than GPU cores 1:11:49 – Brains vs chips 1:15:22 – A GPU is just a bunch of tiny TPUs Look up Dwarkesh Podcast on YouTube/Spotify/etc to watch. Enjoy!

Dwarkesh Patel

931,631 Aufrufe • vor 1 Monat

Episode w Dylan Patel and Asianometry out! A bonanza on how the semiconductor industry actually works. & what Xi could do if he becomes scaling pilled, how we can train models with 10,000x GPT-4's compute by end of decade, & much more. This one is very fun. Enjoy! Links below. 0:00:00 – Xi’s path to AGI 0:05:05 – Liang Mong Song 0:09:10 – How semiconductors get better 0:12:01 – China can centralize compute 0:19:35 – Export controls & sanctions 0:33:45 – Huawei’s intense culture 0:39:45 – Why the semiconductor industry is so stratified 0:41:50 – N2 should not exist 0:46:47 – Taiwan invasion hypothetical 0:50:15 – Mind-boggling complexity of semiconductors 1:00:06 – Chip architecture design 1:05:29 – Architectures lead to different AI models? China vs. US 1:11:06 – Being head of compute at an AI lab 1:17:17 – Scaling costs and power demand 1:37:59 – Are we financing an AI bubble?
2:11:02

Episode w Dylan Patel and Asianometry out! A bonanza on how the semiconductor industry actually works. & what Xi could do if he becomes scaling pilled, how we can train models with 10,000x GPT-4's compute by end of decade, & much more. This one is very fun. Enjoy! Links below. 0:00:00 – Xi’s path to AGI 0:05:05 – Liang Mong Song 0:09:10 – How semiconductors get better 0:12:01 – China can centralize compute 0:19:35 – Export controls & sanctions 0:33:45 – Huawei’s intense culture 0:39:45 – Why the semiconductor industry is so stratified 0:41:50 – N2 should not exist 0:46:47 – Taiwan invasion hypothetical 0:50:15 – Mind-boggling complexity of semiconductors 1:00:06 – Chip architecture design 1:05:29 – Architectures lead to different AI models? China vs. US 1:11:06 – Being head of compute at an AI lab 1:17:17 – Scaling costs and power demand 1:37:59 – Are we financing an AI bubble?

Dwarkesh Patel

260,187 Aufrufe • vor 1 Jahr

Scientist and physician Michael Nehls explains how Alzheimer’s can be reversed. 0:00 Why Dogs Are Healthy for You, Algae Oil, and Lithium 4:48 How Lifestyle Can Affect the Risk of Getting Alzheimer’s 12:22 The Dangers and Lies of mRNA Injections 18:51 The Hidden Power of Lithium 31:24 Brain Fuel From the Sea 37:21 The WEF's Keynote Pushback on Nehls 41:56 How Could You Create a Deficiency in People? 50:27 The Law of Minimum 54:52 The Signs of Lithium Deficiency 1:00:13 What Would Happen if US Citizens Took Lithium Every Day? 1:13:51 Are There Any Negative Effects to Taking Lithium? 1:14:53 How to See Results From Taking Lithium Daily 1:18:30 How to Get Access to Lithium 1:21:35 The Dogma of the Pharmaceutical Industry
1:24:05

Scientist and physician Michael Nehls explains how Alzheimer’s can be reversed. 0:00 Why Dogs Are Healthy for You, Algae Oil, and Lithium 4:48 How Lifestyle Can Affect the Risk of Getting Alzheimer’s 12:22 The Dangers and Lies of mRNA Injections 18:51 The Hidden Power of Lithium 31:24 Brain Fuel From the Sea 37:21 The WEF's Keynote Pushback on Nehls 41:56 How Could You Create a Deficiency in People? 50:27 The Law of Minimum 54:52 The Signs of Lithium Deficiency 1:00:13 What Would Happen if US Citizens Took Lithium Every Day? 1:13:51 Are There Any Negative Effects to Taking Lithium? 1:14:53 How to See Results From Taking Lithium Daily 1:18:30 How to Get Access to Lithium 1:21:35 The Dogma of the Pharmaceutical Industry

Tucker Carlson

739,834 Aufrufe • vor 3 Tagen

New blackboard lecture w Eric Jang He walks through how to build AlphaGo from scratch, but with modern AI tools. Sometimes you understand the future better by stepping backward. AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning from experience, and self-play. You have to go back to 2017 to get insight into how the more general AIs of the future might learn. Once he explained how AlphaGo works, it gave us the context to have a discussion about how RL works in LLMs and how it could work better – naive policy gradient RL has to figure out which of the 100k+ tokens in your trajectory actually got you the right answer, while AlphaGo’s MCTS suggests a strictly better action every single move, giving you a training target that sidesteps the credit assignment problem. The way humans learn is surely closer to the second. Eric also kickstarted an Autoresearch loop on his project. And it was very interesting to discuss which parts of AI research LLMs can already automate pretty well (implementing and running experiments, optimizing hyperparameters) and which they still struggle with (choosing the right question to investigate next, escaping research dead ends). Informative to all the recent discussion about when we should expect an intelligence explosion, and what it would look like from the inside. Timestamps: 0:00:00 – Basics of Go 0:08:06 – Monte Carlo Tree Search 0:31:53 – What the neural network does 1:00:22 – Self-play 1:25:27 – Alternative RL approaches 1:45:36 – Why doesn’t MCTS work for LLMs 2:00:58 – Off-policy training 2:11:51 – RL is even more information inefficient than you thought 2:22:05 – Automated AI researchers
2:37:17

New blackboard lecture w Eric Jang He walks through how to build AlphaGo from scratch, but with modern AI tools. Sometimes you understand the future better by stepping backward. AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning from experience, and self-play. You have to go back to 2017 to get insight into how the more general AIs of the future might learn. Once he explained how AlphaGo works, it gave us the context to have a discussion about how RL works in LLMs and how it could work better – naive policy gradient RL has to figure out which of the 100k+ tokens in your trajectory actually got you the right answer, while AlphaGo’s MCTS suggests a strictly better action every single move, giving you a training target that sidesteps the credit assignment problem. The way humans learn is surely closer to the second. Eric also kickstarted an Autoresearch loop on his project. And it was very interesting to discuss which parts of AI research LLMs can already automate pretty well (implementing and running experiments, optimizing hyperparameters) and which they still struggle with (choosing the right question to investigate next, escaping research dead ends). Informative to all the recent discussion about when we should expect an intelligence explosion, and what it would look like from the inside. Timestamps: 0:00:00 – Basics of Go 0:08:06 – Monte Carlo Tree Search 0:31:53 – What the neural network does 1:00:22 – Self-play 1:25:27 – Alternative RL approaches 1:45:36 – Why doesn’t MCTS work for LLMs 2:00:58 – Off-policy training 2:11:51 – RL is even more information inefficient than you thought 2:22:05 – Automated AI researchers

Dwarkesh Patel

696,173 Aufrufe • vor 1 Monat

New episode with Dr. Konrad Kording (Kording Lab 🦖), professor of bioengineering and neuroscience at the University of Pennsylvania (Penn) and co-director of CIFAR's Learning in Machines & Brains program (CIFAR). Konrad works at the intersection of causality, machine learning, and neuroscience, building rigorous methods for causal reasoning when experiments aren't possible — and challenging how researchers interpret neural data and build AI. Konrad argues the most promising path to understanding how the brain works is to read the brain’s wiring directly, down to the molecular detail of each connection, and to build compilers and simulations to understand the brain’s computation directly. In this episode we go deep into how neurons work, how neurons wire together, and how organic and artificial neural networks differ. We discuss why organic neurons are doing much more; how a model of a single organic neuron can solve MNIST — computing more like a 3-layer artificial neural network; how the brain might learn by solving credit assignment with only local signals; how to approximate backprop without a global algorithm; why AI and humans are intelligent along different dimensions; why Konrad isn’t very worried about AI replacing us; economic models of intelligence and physical work; and much more. Konrad is a brilliant, contrarian thinker who explains complex concepts very intuitively. It is a solid computational neuroscience primer. I hope you enjoy this conversation as much as I did! Other links to this episode and references below. Chapters 00:00:00 Introduction 00:01:01 How organic neurons work 00:24:13 How the brain learns: circuits and credit assignment 00:45:29 Recording the brain 00:52:47 Why simulating brains is hard 01:05:00 A new approach: connectomes and compilers 01:21:00 Why simulate brains? 01:29:50 How AI and human intelligence differ 01:41:04 Evolution, intelligence and AI risk 01:52:42 Robotics, causality, and the roots of intelligence 02:05:53 AI for science and scientific rigor 02:13:05 The economics of intelligence 02:27:50 A hopeful future
2:29:40

New episode with Dr. Konrad Kording (Kording Lab 🦖), professor of bioengineering and neuroscience at the University of Pennsylvania (Penn) and co-director of CIFAR's Learning in Machines & Brains program (CIFAR). Konrad works at the intersection of causality, machine learning, and neuroscience, building rigorous methods for causal reasoning when experiments aren't possible — and challenging how researchers interpret neural data and build AI. Konrad argues the most promising path to understanding how the brain works is to read the brain’s wiring directly, down to the molecular detail of each connection, and to build compilers and simulations to understand the brain’s computation directly. In this episode we go deep into how neurons work, how neurons wire together, and how organic and artificial neural networks differ. We discuss why organic neurons are doing much more; how a model of a single organic neuron can solve MNIST — computing more like a 3-layer artificial neural network; how the brain might learn by solving credit assignment with only local signals; how to approximate backprop without a global algorithm; why AI and humans are intelligent along different dimensions; why Konrad isn’t very worried about AI replacing us; economic models of intelligence and physical work; and much more. Konrad is a brilliant, contrarian thinker who explains complex concepts very intuitively. It is a solid computational neuroscience primer. I hope you enjoy this conversation as much as I did! Other links to this episode and references below. Chapters 00:00:00 Introduction 00:01:01 How organic neurons work 00:24:13 How the brain learns: circuits and credit assignment 00:45:29 Recording the brain 00:52:47 Why simulating brains is hard 01:05:00 A new approach: connectomes and compilers 01:21:00 Why simulate brains? 01:29:50 How AI and human intelligence differ 01:41:04 Evolution, intelligence and AI risk 01:52:42 Robotics, causality, and the roots of intelligence 02:05:53 AI for science and scientific rigor 02:13:05 The economics of intelligence 02:27:50 A hopeful future

Juan Benet

48,748 Aufrufe • vor 11 Tagen

🎙️ Charles Lee: "Huge resolve from this team." "It shows the commitment to togetherness, because I see a lot of times those guys who could be competing for minutes are talking to one another about how they can help not get in foul trouble or how they can be in a better position." "Even for (Xavier Tillman) to come in today and play his 1:12—because we wanted to be in a certain defense—and he's still trying to figure it out, but the guys are talking him through how he can be effective out there." "It's a mindset that the group is developing, and they understand how much they need each other, how much we need them to come in and impact the game in a lot of different ways. So, proud of them."
5:03

🎙️ Charles Lee: "Huge resolve from this team." "It shows the commitment to togetherness, because I see a lot of times those guys who could be competing for minutes are talking to one another about how they can help not get in foul trouble or how they can be in a better position." "Even for (Xavier Tillman) to come in today and play his 1:12—because we wanted to be in a certain defense—and he's still trying to figure it out, but the guys are talking him through how he can be effective out there." "It's a mindset that the group is developing, and they understand how much they need each other, how much we need them to come in and impact the game in a lot of different ways. So, proud of them."

r/CharlotteHornets

11,066 Aufrufe • vor 4 Monaten

🚨Our episode with Anne-Laure Le Cunff is now live! Anne-Laure Le Cunff is the founder of Ness Labs and author of Tiny Experiments: How to Live Freely in a Goal-Obsessed World. She's a neuroscience PhD and writes a newsletter that has 100,000+ subscribers. She's used those insights to develop a new model of success — one built around conducting “tiny experiments” that help her build a life on her own terms. She joins me to discuss how we get trapped in cognitive scripts, the hidden dangers of productivity culture, how we can experiment our way to a better life and MUCH more! Timestamps 0:00:00 Intro 0:00:48 Guest introduction 0:05:15 How do you know you are bored out? 0:17:18 People who love us the most might turn out to be our biggest blockers 0:22:20 Don't confuse activity with effectiveness 0:27:39 We will do virtually anything to gain what is really an illusion of control 0:32:16 The map is not the territory, the menu is not the meal. And yet, words are magic spells 0:39:16 The Winner’s Script and the Loser’s Script 0:47:34 "You gotta run at the top speed if you just want to stay in place.” 0:50:27 Let go of the linear and replace it with the loop- a more cyclical approach for growth 0:55:20 Can you sit alone in a room for 15 minutes? 0:58:24 Procrastination is just a signal from your brain that something is not quite working right now 1:20:38 We know nothing 1:24:26 AI is a rocket ship for the mind 1:28:20 In 100 years, nobody will remember you
1:31:29

🚨Our episode with Anne-Laure Le Cunff is now live! Anne-Laure Le Cunff is the founder of Ness Labs and author of Tiny Experiments: How to Live Freely in a Goal-Obsessed World. She's a neuroscience PhD and writes a newsletter that has 100,000+ subscribers. She's used those insights to develop a new model of success — one built around conducting “tiny experiments” that help her build a life on her own terms. She joins me to discuss how we get trapped in cognitive scripts, the hidden dangers of productivity culture, how we can experiment our way to a better life and MUCH more! Timestamps 0:00:00 Intro 0:00:48 Guest introduction 0:05:15 How do you know you are bored out? 0:17:18 People who love us the most might turn out to be our biggest blockers 0:22:20 Don't confuse activity with effectiveness 0:27:39 We will do virtually anything to gain what is really an illusion of control 0:32:16 The map is not the territory, the menu is not the meal. And yet, words are magic spells 0:39:16 The Winner’s Script and the Loser’s Script 0:47:34 "You gotta run at the top speed if you just want to stay in place.” 0:50:27 Let go of the linear and replace it with the loop- a more cyclical approach for growth 0:55:20 Can you sit alone in a room for 15 minutes? 0:58:24 Procrastination is just a signal from your brain that something is not quite working right now 1:20:38 We know nothing 1:24:26 AI is a rocket ship for the mind 1:28:20 In 100 years, nobody will remember you

Infinite Loops 🎙

22,154 Aufrufe • vor 1 Jahr

.Sergey Levine is one of the world's leading robotics researchers (and co-founder of Physical Intelligence). He thinks fully autonomous robots are much closer than people realize - when I pushed him on a prediction, he said 5 years to robots that can autonomously run your household). The big game changer, in his view, is that we finally have LLMs with common sense and prior knowledge that we can scaffold the robot models around. We get into all the reasons this might not happen, the data flywheel from deployment, architecture, simulation, manufacturing, and much more. Enjoy! 0:00:00 – Timeline to widely deployed autonomous robots 0:22:12 – Why robotics will scale faster than self-driving cars 0:32:15 – How vision-language-action models work 0:50:26 – Improvements needed for brainlike efficiency 1:02:48 – Learning from simulation 1:14:08 – How much will robots speed up AI buildouts? 1:22:54 – If hardware’s the bottleneck, does China win by default?
1:33:14

.Sergey Levine is one of the world's leading robotics researchers (and co-founder of Physical Intelligence). He thinks fully autonomous robots are much closer than people realize - when I pushed him on a prediction, he said 5 years to robots that can autonomously run your household). The big game changer, in his view, is that we finally have LLMs with common sense and prior knowledge that we can scaffold the robot models around. We get into all the reasons this might not happen, the data flywheel from deployment, architecture, simulation, manufacturing, and much more. Enjoy! 0:00:00 – Timeline to widely deployed autonomous robots 0:22:12 – Why robotics will scale faster than self-driving cars 0:32:15 – How vision-language-action models work 0:50:26 – Improvements needed for brainlike efficiency 1:02:48 – Learning from simulation 1:14:08 – How much will robots speed up AI buildouts? 1:22:54 – If hardware’s the bottleneck, does China win by default?

Dwarkesh Patel

210,969 Aufrufe • vor 9 Monaten

We are excited to share that Logan Kilpatrick joined us on The Bench to discuss Google's new Gemini 3.5 Flash: why it's deliberately more persistent and capable than previous Flash models, how it hit #1 on our FinanceAgent Benchmark taking 82 steps where competitors stopped at 13, and what justifies the price increase. We also get into why AI benchmarks need a paradigm shift, the trade-off of building everything vs staying focused, the Pope, and why Omni might kill the Subway Surfers content era. 0:11:00 – Flash is being rebased for the agent era, not just a cheaper model anymore 0:14:03 – Persistence by design: 82 tool calls vs competitors' 13 0:17:52 – Why pricing went up and how Google thinks about value per token 0:22:55 – Coding performance: from 20th to 10th place in one generation 0:28:28 – Why benchmarks have historically been misleading and what the new era of evaluation looks like 0:29:28 Logan on why Google has the best researchers in the world 0:36:16 – The cost of being Google 0:39:07 – The Pope’s encyclical on AI and whether most people see frontier intelligence as a good thing 0:51:12 – Why Omni is the thing that recently clicked
54:53

We are excited to share that Logan Kilpatrick joined us on The Bench to discuss Google's new Gemini 3.5 Flash: why it's deliberately more persistent and capable than previous Flash models, how it hit #1 on our FinanceAgent Benchmark taking 82 steps where competitors stopped at 13, and what justifies the price increase. We also get into why AI benchmarks need a paradigm shift, the trade-off of building everything vs staying focused, the Pope, and why Omni might kill the Subway Surfers content era. 0:11:00 – Flash is being rebased for the agent era, not just a cheaper model anymore 0:14:03 – Persistence by design: 82 tool calls vs competitors' 13 0:17:52 – Why pricing went up and how Google thinks about value per token 0:22:55 – Coding performance: from 20th to 10th place in one generation 0:28:28 – Why benchmarks have historically been misleading and what the new era of evaluation looks like 0:29:28 Logan on why Google has the best researchers in the world 0:36:16 – The cost of being Google 0:39:07 – The Pope’s encyclical on AI and whether most people see frontier intelligence as a good thing 0:51:12 – Why Omni is the thing that recently clicked

Vals AI

16,486 Aufrufe • vor 1 Monat

Goldman Sachs CEO David Solomon on the open question of agentic investing: "Models work on the facts that are available." "Sometimes the biggest changes in the way you have to think about investing in a portfolio comes from things that are completely new and unexpected." "It can't be something from the past." "There are a handful of people who have so outperformed as investors over a long period of time, but... generally speaking, people underperform." "And so if the models are based on the information that the people are underperforming all have, it's going to be interesting to see whether something different comes out of it." David Solomon with David Haber and benahorowitz.eth
0:58

Goldman Sachs CEO David Solomon on the open question of agentic investing: "Models work on the facts that are available." "Sometimes the biggest changes in the way you have to think about investing in a portfolio comes from things that are completely new and unexpected." "It can't be something from the past." "There are a handful of people who have so outperformed as investors over a long period of time, but... generally speaking, people underperform." "And so if the models are based on the information that the people are underperforming all have, it's going to be interesting to see whether something different comes out of it." David Solomon with David Haber and benahorowitz.eth

a16z

62,860 Aufrufe • vor 5 Monaten

Just in $AMD Anush "Speed is the moat"|ROCm🎙️ In the race to define the future of AI, what's the one advantage that truly lasts? It's not proprietary tech, argues Anush Elangovan Elangovan, VP of AI Software at AMD , but the sustainable speed of innovation. He explains why AMD is rejecting the "walled garden" model for its open source ROCm stack, betting that an open community flywheel is the key to victory. Listen to understand how this open strategy is designed to out-innovate closed systems by empowering developers to solve everything from frontier-model challenges to the mundane, everyday problems that define the "last mile" of AI. AMD ROCm Software: Part 1 Transcript [00:00:00] Andrew Zigler: Joining me is Anush Elangovan, VP of AI software at AMD. And when people talk about AI compute, the conversation often stops at hardware specs, but it's more than just physical chips that win the game. It's also the software ecosystems supporting them. [00:00:18] Andrew Zigler: The prevailing strategy in the industry has been to build something like a walled garden. You know, something closed, proprietary locks, developers in. But AMD is betting on an entirely different play, open source acceleration, and with rock, their open source AI software stack. AMD is building not just hardware parity, but an innovation flywheel that's powered by the community with interoperability and the freedom to scale without all of that pesky lockin. [00:00:48] Andrew Zigler: And in this world, speed is your moat and how fast you can innovate while your platform remains open, flexible, and standardize across all of its applications. That's what we're gonna explore [00:01:00] today. So Anush, I'm really excited to have you here. Welcome to Dev Interrupted. [00:01:04] Anush Elangovan: Thanks for having me. Uh, super excited to chat about it. [00:01:07] Andrew Zigler: Amazing. Well, let's go ahead and dive right in with kind of what I laid it out with in the beginning, the idea of the moat and it being about speed. I wanna unpack that a bit because that came from you when you and I first spoke. And I, and I want to know, you know, how do you define speed inside of AMD beyond just things like hardware, benchmarks. [00:01:27] Anush Elangovan: Yeah, that's a very good question. So when we typically talk about speed, everyone's like, Hey, hardware benchmark specs, right? Like, uh, memory bandwidth or, or flops. And that is one important part of it, uh, AMD does very well. With that, we do have, a, a very good history of executing on that axis. [00:01:47] Anush Elangovan: But when I say speed is the moat, it is about, uh, how we prepare, how we build the muscle to run the race for a long time and run it fast. And it is [00:02:00] not about a single point in time that you've, you've beat some you know, benchmark and, and you declare victory. It's about building the ability to consistently develop and deliver. [00:02:13] Anush Elangovan: Both hardware and software innovation at scale and do it fast, right? Like, you know, we we're increasingly getting to a point where models come out and they're, uh, you know, a year or two ago it was like, Hey, they work on AMD on day zero, which is great, but now they are performing on AMD the day it releases, right? [00:02:32] Anush Elangovan: So, what does it take to Prefetch where the industry is going? Be prepared to intercept. At that point is what you know, I, I refer to as you know, the, the speed factor in, in creating this mode, right? And the mode is just shed all things that hold you back and run as fast as you can. [00:02:53] Anush Elangovan: Uh, because the pace of innovation that is, uh, being seen in, in AI [00:03:00] industries is just. Amazing. Right? And it's like, it's transformational at at how you generate electricity. It's transformational as at how you build data centers. It's transformational at how you deploy compute, networking. It's transformational at what kind of use cases you, you know, uh, use AI for. [00:03:17] Anush Elangovan: Uh, and for that, you need to be prepared to, see what comes tomorrow and be prepared to run the race tomorrow. [00:03:23] Andrew Zigler: Yeah, it's a really great perspective because it highlights that it's not just like a checkpoint that you run through. I like how you called out, like it's not just hitting that benchmark or being the best in class at that moment, in that snapshot, it's about having a. The throughput and about having that dedication to the idea and continuing to deliver on it. [00:03:43] Andrew Zigler: It's not just crossing the threshold, but it's also being the engine. And that's what, that's what protects a business. That is the moat, because the moat is that innovation layer, the faster and more, uh, future forward. That you can work and think, [00:04:00] you know, the better. Uh, we, we talk a lot about like future forward work styles. [00:04:04] Andrew Zigler: Like what are the things I could be doing right now today that are gonna be like, way more useful tomorrow? Let, let's abandon those, workflows that are older and that kind of like, that translates into. An advantage when you work that way. You know, what kind of things have you learned working with, uh, like across all spectrums of people who would use ROCm, right? [00:04:23] Andrew Zigler: You have like the developers, but then you also have the enterprises and you have this large span of adoptees, right? So what is the, what does that look like that you learn? [00:04:32] Anush Elangovan: Yeah, so, so the way I look at it is there are gonna be pockets of different, uh, you know, cadences, right? Like, so people who are deploying in enterprises, for example, right? The validation and how long it takes for them to deploy an LLM that's secure. It's, with guardrails, et cetera, maybe longer. [00:04:52] Anush Elangovan: but you still have to go through the process and you have to be prepared to like, walk that walk to deploy an enterprises. That doesn't mean it's [00:05:00] not fast, that's as fast as you can do for that industry, right? And if you are deploying AI in healthcare, right, it's, it's got its own, uh, cycle. [00:05:07] Anush Elangovan: but in each one of these, you want to see how, like, go down to the essence of what is it that you actually have to do. And, you know, I, I, I like how you framed it. It's like it's, you shed your prior assumptions of how things are done, right. And, and you kind of build up from a, uh, first principles, uh, approach to say, this is how I could use AI to unlock, whatever I'm doing. [00:05:33] Anush Elangovan: And, and, some of it, you know, it's good to really step back and look at. Just question every part of it, right? Like right now you're getting chat GPT and, Gemini competing for like, math, olympiads and, and, uh, college, uh, reasoning, uh, tests. Right? And, and those are like that, that is amazing and increasingly like complex tasks that they're trying to do. [00:05:58] Anush Elangovan: But there may also be like. [00:06:00] More mundane things that AI could, could get applied to. Right? And, and so when we think about shedding old ways, you wanna shed it not just in like the tip of the spear. It's like, you know, I'm gonna see what's the frontier model. It's also, it could be something as simple as. [00:06:18] Anush Elangovan: How do you choose a, a movie, uh, you know, like a recommendation system, right? Or, or, uh, an automated, uh, flight, uh, rebooking system. So the moment, you know, your flight is late, uh, right now it's a notification, right? It's like, oh, you got a text message saying your flight's late. And I got that like three times this week. [00:06:38] Anush Elangovan: But anyway, uh, and, and, and, and, I was just like, okay, so if I were to rethink this. All this MCPs that we have that should be hooked up into an MCP that says, your flight's delayed. Here are your options. If you want, you know, these are the paid options. Yeah. Here are the free options. This will get you back into your you know, Toronto airport [00:07:00] tonight. [00:07:00] Anush Elangovan: Or if you stay, here's a hotel plus this, plus this, plus. It's just like, go ahead is all I should say. Versus now I'm like, okay, can someone, you know, can I call a travel agent? Can I do this? Can I go online and log into And you know, so we gotta fundamentally rethink even those like small, nuances of, things that we do that can be automated out and AI is really, really good at doing something like this, right? Maybe I just explained an AI startup idea right now. Somebody should just start that. [00:07:29] Andrew Zigler: I think you did. Yeah, you definitely did. Someone, one of our listeners is definitely going to lift that off of you. I, I, I, you know, I hate being on the receiving end of those. You feel a little helpless and then you have to like, follow the whole flow. So I know what you mean. Like I, I like how you called out that the build and this like. [00:07:45] Andrew Zigler: Where speed is your moat and the innovation layer is protecting you, is what makes you better than your competitors. How you scale that and you bring that to market. So by understanding the problems that you're solving, uh, throwing away those older assumptions, but also [00:08:00] recognizing that like. We're building every single day, new things and new ways of using stuff that we're still figuring out the implications of. [00:08:08] Andrew Zigler: And so when you have a lot of velocity and you're introducing a lot of new ideas, and maybe you have that workflow now that automatically rebook your flight off of your late flight text message, and uh, I know I would certainly use it, but you know, what kind of philosophies guide the way that y'all think about building this ecosystem to manage that stability while letting folks. [00:08:29] Andrew Zigler: Play with the speed and the assumptions and the airplane re bookings. [00:08:34] Anush Elangovan: so, so I think, you know, we need to peel one layer down, right? and the philosophy is, Hey, we, we just discovered electricity, right? And you know what we're gonna do? We are gonna make motors, uh, or dynamos, right? Like engines. Uh, sure. We don't know if it's gonna be a Ferrari that you're gonna make, or it's a a a a dump truck. [00:08:57] Anush Elangovan: That's good for doing this. But let's [00:09:00] let, which is also required, right? You need a dump truck. You need a garbage truck. And, [00:09:04] Andrew Zigler: Yeah. You need the [00:09:04] Anush Elangovan: course you need, uh, a Ferrari for a midlife crisis, right? So, [00:09:09] Andrew Zigler: precisely. [00:09:10] Anush Elangovan: But, but my, uh, point is what do we build next? And, uh, and this is what I meant by like, okay, let's, let's take those baby steps to build the. [00:09:20] Anush Elangovan: Infrastructure that's required that we know we'll have to use, right? So, so if I just discovered electricity, okay, great. Now one, how do I save this electricity and how do I use it? So there's battery technology, so you need to do something like that, right? Like so. But then you also want to make it into an actionable thing. [00:09:37] Anush Elangovan: You want to make it for like automobiles, or you wanna use it for, you know, powering, uh, entire cities. So it is that transformational. So, uh, AI is that transformational. So, if you distill down, it'll, it'll come down to how do we think about, what we can do with this this fundamental technology that, We may not be aware of what it [00:10:00] is gonna unlock next, but at least you know the next step is clear, right? It's like a dense fog, you know, it's gonna be like, it, it's the right path. You see the light, but it's kind of like out there and, and the steps you're taking are concrete and you're like, okay, this is good. [00:10:16] Anush Elangovan: I, this is better than where I was or where we were. So we are moving forward. So you can build with the. Intuition from what you see in the short term and a tactical view, but towards what you think the future is gonna be. [00:10:28] Andrew Zigler: Right. You almost like we're all in this like fog of war, right? And like you said, you're reaching out and you're trying to step through it. You could think of it too, as like you're in the dark and your hands are up in front of you and you know that. You're, you're not gonna run your face into a wall because your hands are out in front of you, but you're not gonna maybe do much better than that. [00:10:45] Andrew Zigler: So that's kind of like, I think the eco, the, the industry, the world that we find ourselves in, uh, and we all have to, then this becomes the power of an ecosystem, of a group of people working together to create that layer of, [00:11:00] uh, of establishing the [00:11:01] Anush Elangovan: exactly. And I, I, I just, instead of, you know, saying fog of war I describe it as like, you're in this. Beautiful valley with like a morning, uh, fog that's in. You can smell the flowers. You, you hear the birds. You are like, okay, it's, we are in like, uh, utopian paradise and yes, I just need to like, continue the walk, right? [00:11:24] Anush Elangovan: and then move forward with that, conviction that you're in the right spot. [00:11:27] Andrew Zigler: Yeah. So let's talk about that ecosystem world. This nice, I love how you describe it, this grassy side of a hill in the morning that's covered in some mist and maybe we can't see 30 feet in one direction, but it sure is a beautiful hill and it smells nice. And so we're all here. And why is, in that world, why is. [00:11:44] Andrew Zigler: You know, open source, their strategic advantage that y'all are going for in the AI hardware market. And, and then how does like ROCm turn that into wins for people within that ecosystem? [00:11:56] Anush Elangovan: you know, the, the way we look at it is this, is kind of like how I view [00:12:00] AI and the ecosystem, right? But, but it is for everyone to enjoy. Uh, and so we do want to make sure that. You know, it is, uh, beneficial for everyone. [00:12:09] Anush Elangovan: The ecosystem can come in and, and innovate. It's an open innovation engine. and uh, it is very different from, you know, having a walled garden with, Hey, only I know how to do this and I'm gonna do it and throw it over the fence and you can use it or keep walking, right? So we'd like to be good citizens that way, but also. [00:12:30] Anush Elangovan: Uh, it is self-fulfilling in a way, right? Like it, the, the pace at which we innovate with open source is unmatched. Like, you know, our serving engines are like VLLM and, and sg l. Those things, uh, those frameworks are like super, super aggressive in terms of how fast they come out with features and how fast they can you know, get performant models out. [00:12:52] Anush Elangovan: And that compared with what, uh, you'd get from, you know, the likes of like T-R-T-L-L-M or something is always lagging, right? Because you [00:13:00] just can't keep up with you know, 200 commits a week just on one particular model to get that model really performant [00:13:06] Andrew Zigler: And, and, and in that world where, you know, everyone can enjoy the winds of this, what kind of customer stories or innovation stories have really stood out to you and excite you about building and creating this place for developers? [00:13:19] Anush Elangovan: Yeah. So I think the parts that are super exciting for me are when when we get to see a customer that is first skeptical. Then they start a little like, okay, fine, we'll give you a chance. Uh, we do a simple, uh, POC and then they're like, huh, this seems to work. Yeah, we told you it works. [00:13:42] Anush Elangovan: You don't have to change one line of code. Really? Yes, no need to change one line of code. Okay, let's try a production workload. So then they try it. Oh, you're more performant than the competition. Yes. We're more performant than, than the competition. So how much does it cost? And we're like, oh, it's your TCO is better with, uh, [00:14:00] AMD. [00:14:00] Anush Elangovan: So again, they're like, wow, okay, good. So now how do we deploy at scale? And then we go deploy it at scale. And when they give a thumbs up on that and they say, this is good, right? That's when you know, you, you see it go full circle from like, oh, we, we've never heard about AMD to like actually deploy to tens of thousands of GPUs In the order of a few months, right? It, it, it really is fascinating to see and very exciting and invigorating to [00:14:28] Andrew Zigler: Yeah. At like a great exposure to a lot of interesting problems. And, and then people using the infrastructure, the, the technology available to solve those problems. Really specific problems by the way, that's often why they're bringing their data and AI to it, uh, is because it is really specific and important for them. [00:14:45] Andrew Zigler: And there's a, a lot I think that other engineering orgs can learn and even emulate from AMD's success and, and having this open source ecosystem and it causing this acceleration within. You [00:15:00] know, uh, customers and enterprises that use and adopt the tools and, and, and that creates an advantage. And that goes back to why we're talking and like the real thesis of our conversation today. [00:15:10] Andrew Zigler: So how do you think engineering leaders that are listening to this and obviously tapping into this great success AMD has from an open source flywheel, how do you think other, other folks building in the same space can foster that open, first, that open source oriented culture in order to, you know, accelerate their innovation goals? [00:15:29] Anush Elangovan: Yeah, that's a very good question. So the startup that um, was acquired by AMD we, we built, I mean, we started off doing iot stuff and you know, smart ring and all that, right? But in the, the end of like, uh, and not the end, the last six years of the company was building ML compilers. [00:15:47] Anush Elangovan: And ml, ML compilers are like super, uh, complicated, sophisticated, advanced algorithms, dah, dah, dah. but it was all open source, right? So our VCs were like, wait, what do you mean your core [00:16:00] IP is open source? And um, the speed is the moat applied even then, right? It was just like, yes, if you have an idea that. [00:16:08] Anush Elangovan: Because someone saw this idea that you are, they're gonna be able to catch up, then you probably have the wrong idea anyway. But if they are, you know, you execute and they're gonna catch up, that you should assume they're gonna catch up. Right? So you gotta move forward. So keeping it open source is super important. [00:16:25] Anush Elangovan: But also to your question on like, you know, the learnings from an AMD standpoint, right? If there are, hard problems, I'd say dig in and work through it, right? Like there's no way but through it, right? That should be the simple mentality. And more, uh, frequently than not. you'll see that you'll just make it through in a, in, in good form. [00:16:52] Anush Elangovan: But if you doubt it and you're like, oh, I don't know if I should commit, if I'm, I, you know, what should just commit to do the right thing [00:17:00] every step, right? Every step, and just keep taking one step in front of the other. And in no time you'll see that you'll be running. Right. And, and yes, the first few steps will be like, yeah, everyone's complaining about your software quality. [00:17:15] Anush Elangovan: Everyone's complaining about this and that, and it doesn't work. And, and a few steps in, you know, you get, you get the hang of all the complaints that are coming in. You get the feedback loop. You're like, okay, what, what are you prioritizing again? One step in front of the other, right? You just keep knocking that out and then you get to a point where you're, it just becomes second nature, right? To do the, to do the right thing. And, and then yes, if someone gives you two options, you'll be like, fine. This is, uh, you know, there's always the resource trade off. There's always a human capital trade off, but what's the right thing to do? of course, I, I'm pragmatic about what we choose, but, but if the right thing for your long-term success is dig in, go first, principles, make it [00:18:00] happen. [00:18:00] Anush Elangovan: Well. Then just go for that. There's, there is no shortcut to [00:18:04] Andrew Zigler: acknowledging, you know, how it aligns with your mission, your core company goals, and what you're looking to achieve. And, and I, I love how you rightfully called out that in the open source world and you know, you have your technology that you've built, what you think is your moat upon, right? [00:18:22] Andrew Zigler: It's your code and, and to open source that, or to just make it where anyone could peer in is, you know. Scary in one regard, but two, it just kind of feels like you're handing away your throne room in some kind of sense, a very direct feeling sense. But the ultimately, you were really right to call out, and this is something I think about all the time, that the real power there is still the speed This the speed. [00:18:42] Andrew Zigler: That was the moat at the beginning of our conversation. It's the speed in combination with your. Very specific domain understanding of what you're building and what you're creating, and your new role as the steward of that world and how people plug into it, which [00:19:00] has frankly, a lot more influence and power than lording over a closed. [00:19:04] Andrew Zigler: You know, repository or an ecosystem, and like you said, like throwing things over the wall. Sure. There, there might be people always on the other side of that wall, but you're not gonna have a great connection with them. You're not gonna be able to really clearly understand them. I, I like your metaphor of the side of the field of the mountain a lot more. [00:19:23] Andrew Zigler: But, but in the, in this world, you know, where. That speed is, is the power and, and open source is just one way that you can harness that speed to get really far ahead and to innovate. , There's other parts of this equation that you can be experimenting with too, and I'd love to pick your brain about them as a software leader and, and, and one of them is about looking forward and kind of understanding that future that we're all building towards and beyond today's models and hardware. [00:19:48] Andrew Zigler: You know, what do you see as the next major bottleneck or opportunity in the AI compute space? As, as you know, enterprises and folks start to get a little more mature about what's available to [00:20:00] them. [00:20:00] Anush Elangovan: Yeah, I think, the bottleneck and opportunity is, uh, what I'd call, call walking the last mile of ai. Right. Uh, and like I I, I gave you an example, uh, previously, but, but it's similar to that. It's like there are cases where Humans have so many, uh, things to do in your day. You know, like the, if we sit down and actually had a customer focus like, okay, these customers lives, I'm gonna save four hours of this customer's life. And if you actually sit down and look at all of that, it'll be. Easily automatable, easily you know, uh, applicable, uh, for ai, right? [00:20:39] Anush Elangovan: Like, but then making it happen is gonna take a little bit, right? It's like maybe it's, uh, paying your utility bill, right? Or something like that, right? Or, or, your healthcare explanation of benefits. Uh, like, I'm sure you get an explanation of benefits, and I'm like, I, I don't even know what that thing is. [00:20:55] Anush Elangovan: It's just like EOB and like. [00:20:57] Andrew Zigler: it's a big, a big old PDF. Yeah, [00:21:00] exactly. [00:21:01] Anush Elangovan: Like, like, I'm like great straight to the, uh, shredder, right? And but that could be, you know, automated with the ai, right? It, it, it'd be like, Hey, the summary of this thing is you went and visited this day. Everything is okay. Everything is paid for, so don't worry, it's not a bill. [00:21:17] Anush Elangovan: That again, the same, uh, thing, but the sense of what that information overload is could be. Digested by ai, uh, accumulated over time and retrieved when you need it. Like, I don't, I actually don't even need to know this EOB right now, unless of course, whenever I need to know it, that maybe, you know, like for some benefits I need to figure out what do, what did I do over the past year and how do I apply it? Source:
31:35

Just in $AMD Anush "Speed is the moat"|ROCm🎙️ In the race to define the future of AI, what's the one advantage that truly lasts? It's not proprietary tech, argues Anush Elangovan Elangovan, VP of AI Software at AMD , but the sustainable speed of innovation. He explains why AMD is rejecting the "walled garden" model for its open source ROCm stack, betting that an open community flywheel is the key to victory. Listen to understand how this open strategy is designed to out-innovate closed systems by empowering developers to solve everything from frontier-model challenges to the mundane, everyday problems that define the "last mile" of AI. AMD ROCm Software: Part 1 Transcript [00:00:00] Andrew Zigler: Joining me is Anush Elangovan, VP of AI software at AMD. And when people talk about AI compute, the conversation often stops at hardware specs, but it's more than just physical chips that win the game. It's also the software ecosystems supporting them. [00:00:18] Andrew Zigler: The prevailing strategy in the industry has been to build something like a walled garden. You know, something closed, proprietary locks, developers in. But AMD is betting on an entirely different play, open source acceleration, and with rock, their open source AI software stack. AMD is building not just hardware parity, but an innovation flywheel that's powered by the community with interoperability and the freedom to scale without all of that pesky lockin. [00:00:48] Andrew Zigler: And in this world, speed is your moat and how fast you can innovate while your platform remains open, flexible, and standardize across all of its applications. That's what we're gonna explore [00:01:00] today. So Anush, I'm really excited to have you here. Welcome to Dev Interrupted. [00:01:04] Anush Elangovan: Thanks for having me. Uh, super excited to chat about it. [00:01:07] Andrew Zigler: Amazing. Well, let's go ahead and dive right in with kind of what I laid it out with in the beginning, the idea of the moat and it being about speed. I wanna unpack that a bit because that came from you when you and I first spoke. And I, and I want to know, you know, how do you define speed inside of AMD beyond just things like hardware, benchmarks. [00:01:27] Anush Elangovan: Yeah, that's a very good question. So when we typically talk about speed, everyone's like, Hey, hardware benchmark specs, right? Like, uh, memory bandwidth or, or flops. And that is one important part of it, uh, AMD does very well. With that, we do have, a, a very good history of executing on that axis. [00:01:47] Anush Elangovan: But when I say speed is the moat, it is about, uh, how we prepare, how we build the muscle to run the race for a long time and run it fast. And it is [00:02:00] not about a single point in time that you've, you've beat some you know, benchmark and, and you declare victory. It's about building the ability to consistently develop and deliver. [00:02:13] Anush Elangovan: Both hardware and software innovation at scale and do it fast, right? Like, you know, we we're increasingly getting to a point where models come out and they're, uh, you know, a year or two ago it was like, Hey, they work on AMD on day zero, which is great, but now they are performing on AMD the day it releases, right? [00:02:32] Anush Elangovan: So, what does it take to Prefetch where the industry is going? Be prepared to intercept. At that point is what you know, I, I refer to as you know, the, the speed factor in, in creating this mode, right? And the mode is just shed all things that hold you back and run as fast as you can. [00:02:53] Anush Elangovan: Uh, because the pace of innovation that is, uh, being seen in, in AI [00:03:00] industries is just. Amazing. Right? And it's like, it's transformational at at how you generate electricity. It's transformational as at how you build data centers. It's transformational at how you deploy compute, networking. It's transformational at what kind of use cases you, you know, uh, use AI for. [00:03:17] Anush Elangovan: Uh, and for that, you need to be prepared to, see what comes tomorrow and be prepared to run the race tomorrow. [00:03:23] Andrew Zigler: Yeah, it's a really great perspective because it highlights that it's not just like a checkpoint that you run through. I like how you called out, like it's not just hitting that benchmark or being the best in class at that moment, in that snapshot, it's about having a. The throughput and about having that dedication to the idea and continuing to deliver on it. [00:03:43] Andrew Zigler: It's not just crossing the threshold, but it's also being the engine. And that's what, that's what protects a business. That is the moat, because the moat is that innovation layer, the faster and more, uh, future forward. That you can work and think, [00:04:00] you know, the better. Uh, we, we talk a lot about like future forward work styles. [00:04:04] Andrew Zigler: Like what are the things I could be doing right now today that are gonna be like, way more useful tomorrow? Let, let's abandon those, workflows that are older and that kind of like, that translates into. An advantage when you work that way. You know, what kind of things have you learned working with, uh, like across all spectrums of people who would use ROCm, right? [00:04:23] Andrew Zigler: You have like the developers, but then you also have the enterprises and you have this large span of adoptees, right? So what is the, what does that look like that you learn? [00:04:32] Anush Elangovan: Yeah, so, so the way I look at it is there are gonna be pockets of different, uh, you know, cadences, right? Like, so people who are deploying in enterprises, for example, right? The validation and how long it takes for them to deploy an LLM that's secure. It's, with guardrails, et cetera, maybe longer. [00:04:52] Anush Elangovan: but you still have to go through the process and you have to be prepared to like, walk that walk to deploy an enterprises. That doesn't mean it's [00:05:00] not fast, that's as fast as you can do for that industry, right? And if you are deploying AI in healthcare, right, it's, it's got its own, uh, cycle. [00:05:07] Anush Elangovan: but in each one of these, you want to see how, like, go down to the essence of what is it that you actually have to do. And, you know, I, I, I like how you framed it. It's like it's, you shed your prior assumptions of how things are done, right. And, and you kind of build up from a, uh, first principles, uh, approach to say, this is how I could use AI to unlock, whatever I'm doing. [00:05:33] Anush Elangovan: And, and, some of it, you know, it's good to really step back and look at. Just question every part of it, right? Like right now you're getting chat GPT and, Gemini competing for like, math, olympiads and, and, uh, college, uh, reasoning, uh, tests. Right? And, and those are like that, that is amazing and increasingly like complex tasks that they're trying to do. [00:05:58] Anush Elangovan: But there may also be like. [00:06:00] More mundane things that AI could, could get applied to. Right? And, and so when we think about shedding old ways, you wanna shed it not just in like the tip of the spear. It's like, you know, I'm gonna see what's the frontier model. It's also, it could be something as simple as. [00:06:18] Anush Elangovan: How do you choose a, a movie, uh, you know, like a recommendation system, right? Or, or, uh, an automated, uh, flight, uh, rebooking system. So the moment, you know, your flight is late, uh, right now it's a notification, right? It's like, oh, you got a text message saying your flight's late. And I got that like three times this week. [00:06:38] Anush Elangovan: But anyway, uh, and, and, and, and, I was just like, okay, so if I were to rethink this. All this MCPs that we have that should be hooked up into an MCP that says, your flight's delayed. Here are your options. If you want, you know, these are the paid options. Yeah. Here are the free options. This will get you back into your you know, Toronto airport [00:07:00] tonight. [00:07:00] Anush Elangovan: Or if you stay, here's a hotel plus this, plus this, plus. It's just like, go ahead is all I should say. Versus now I'm like, okay, can someone, you know, can I call a travel agent? Can I do this? Can I go online and log into And you know, so we gotta fundamentally rethink even those like small, nuances of, things that we do that can be automated out and AI is really, really good at doing something like this, right? Maybe I just explained an AI startup idea right now. Somebody should just start that. [00:07:29] Andrew Zigler: I think you did. Yeah, you definitely did. Someone, one of our listeners is definitely going to lift that off of you. I, I, I, you know, I hate being on the receiving end of those. You feel a little helpless and then you have to like, follow the whole flow. So I know what you mean. Like I, I like how you called out that the build and this like. [00:07:45] Andrew Zigler: Where speed is your moat and the innovation layer is protecting you, is what makes you better than your competitors. How you scale that and you bring that to market. So by understanding the problems that you're solving, uh, throwing away those older assumptions, but also [00:08:00] recognizing that like. We're building every single day, new things and new ways of using stuff that we're still figuring out the implications of. [00:08:08] Andrew Zigler: And so when you have a lot of velocity and you're introducing a lot of new ideas, and maybe you have that workflow now that automatically rebook your flight off of your late flight text message, and uh, I know I would certainly use it, but you know, what kind of philosophies guide the way that y'all think about building this ecosystem to manage that stability while letting folks. [00:08:29] Andrew Zigler: Play with the speed and the assumptions and the airplane re bookings. [00:08:34] Anush Elangovan: so, so I think, you know, we need to peel one layer down, right? and the philosophy is, Hey, we, we just discovered electricity, right? And you know what we're gonna do? We are gonna make motors, uh, or dynamos, right? Like engines. Uh, sure. We don't know if it's gonna be a Ferrari that you're gonna make, or it's a a a a dump truck. [00:08:57] Anush Elangovan: That's good for doing this. But let's [00:09:00] let, which is also required, right? You need a dump truck. You need a garbage truck. And, [00:09:04] Andrew Zigler: Yeah. You need the [00:09:04] Anush Elangovan: course you need, uh, a Ferrari for a midlife crisis, right? So, [00:09:09] Andrew Zigler: precisely. [00:09:10] Anush Elangovan: But, but my, uh, point is what do we build next? And, uh, and this is what I meant by like, okay, let's, let's take those baby steps to build the. [00:09:20] Anush Elangovan: Infrastructure that's required that we know we'll have to use, right? So, so if I just discovered electricity, okay, great. Now one, how do I save this electricity and how do I use it? So there's battery technology, so you need to do something like that, right? Like so. But then you also want to make it into an actionable thing. [00:09:37] Anush Elangovan: You want to make it for like automobiles, or you wanna use it for, you know, powering, uh, entire cities. So it is that transformational. So, uh, AI is that transformational. So, if you distill down, it'll, it'll come down to how do we think about, what we can do with this this fundamental technology that, We may not be aware of what it [00:10:00] is gonna unlock next, but at least you know the next step is clear, right? It's like a dense fog, you know, it's gonna be like, it, it's the right path. You see the light, but it's kind of like out there and, and the steps you're taking are concrete and you're like, okay, this is good. [00:10:16] Anush Elangovan: I, this is better than where I was or where we were. So we are moving forward. So you can build with the. Intuition from what you see in the short term and a tactical view, but towards what you think the future is gonna be. [00:10:28] Andrew Zigler: Right. You almost like we're all in this like fog of war, right? And like you said, you're reaching out and you're trying to step through it. You could think of it too, as like you're in the dark and your hands are up in front of you and you know that. You're, you're not gonna run your face into a wall because your hands are out in front of you, but you're not gonna maybe do much better than that. [00:10:45] Andrew Zigler: So that's kind of like, I think the eco, the, the industry, the world that we find ourselves in, uh, and we all have to, then this becomes the power of an ecosystem, of a group of people working together to create that layer of, [00:11:00] uh, of establishing the [00:11:01] Anush Elangovan: exactly. And I, I, I just, instead of, you know, saying fog of war I describe it as like, you're in this. Beautiful valley with like a morning, uh, fog that's in. You can smell the flowers. You, you hear the birds. You are like, okay, it's, we are in like, uh, utopian paradise and yes, I just need to like, continue the walk, right? [00:11:24] Anush Elangovan: and then move forward with that, conviction that you're in the right spot. [00:11:27] Andrew Zigler: Yeah. So let's talk about that ecosystem world. This nice, I love how you describe it, this grassy side of a hill in the morning that's covered in some mist and maybe we can't see 30 feet in one direction, but it sure is a beautiful hill and it smells nice. And so we're all here. And why is, in that world, why is. [00:11:44] Andrew Zigler: You know, open source, their strategic advantage that y'all are going for in the AI hardware market. And, and then how does like ROCm turn that into wins for people within that ecosystem? [00:11:56] Anush Elangovan: you know, the, the way we look at it is this, is kind of like how I view [00:12:00] AI and the ecosystem, right? But, but it is for everyone to enjoy. Uh, and so we do want to make sure that. You know, it is, uh, beneficial for everyone. [00:12:09] Anush Elangovan: The ecosystem can come in and, and innovate. It's an open innovation engine. and uh, it is very different from, you know, having a walled garden with, Hey, only I know how to do this and I'm gonna do it and throw it over the fence and you can use it or keep walking, right? So we'd like to be good citizens that way, but also. [00:12:30] Anush Elangovan: Uh, it is self-fulfilling in a way, right? Like it, the, the pace at which we innovate with open source is unmatched. Like, you know, our serving engines are like VLLM and, and sg l. Those things, uh, those frameworks are like super, super aggressive in terms of how fast they come out with features and how fast they can you know, get performant models out. [00:12:52] Anush Elangovan: And that compared with what, uh, you'd get from, you know, the likes of like T-R-T-L-L-M or something is always lagging, right? Because you [00:13:00] just can't keep up with you know, 200 commits a week just on one particular model to get that model really performant [00:13:06] Andrew Zigler: And, and, and in that world where, you know, everyone can enjoy the winds of this, what kind of customer stories or innovation stories have really stood out to you and excite you about building and creating this place for developers? [00:13:19] Anush Elangovan: Yeah. So I think the parts that are super exciting for me are when when we get to see a customer that is first skeptical. Then they start a little like, okay, fine, we'll give you a chance. Uh, we do a simple, uh, POC and then they're like, huh, this seems to work. Yeah, we told you it works. [00:13:42] Anush Elangovan: You don't have to change one line of code. Really? Yes, no need to change one line of code. Okay, let's try a production workload. So then they try it. Oh, you're more performant than the competition. Yes. We're more performant than, than the competition. So how much does it cost? And we're like, oh, it's your TCO is better with, uh, [00:14:00] AMD. [00:14:00] Anush Elangovan: So again, they're like, wow, okay, good. So now how do we deploy at scale? And then we go deploy it at scale. And when they give a thumbs up on that and they say, this is good, right? That's when you know, you, you see it go full circle from like, oh, we, we've never heard about AMD to like actually deploy to tens of thousands of GPUs In the order of a few months, right? It, it, it really is fascinating to see and very exciting and invigorating to [00:14:28] Andrew Zigler: Yeah. At like a great exposure to a lot of interesting problems. And, and then people using the infrastructure, the, the technology available to solve those problems. Really specific problems by the way, that's often why they're bringing their data and AI to it, uh, is because it is really specific and important for them. [00:14:45] Andrew Zigler: And there's a, a lot I think that other engineering orgs can learn and even emulate from AMD's success and, and having this open source ecosystem and it causing this acceleration within. You [00:15:00] know, uh, customers and enterprises that use and adopt the tools and, and, and that creates an advantage. And that goes back to why we're talking and like the real thesis of our conversation today. [00:15:10] Andrew Zigler: So how do you think engineering leaders that are listening to this and obviously tapping into this great success AMD has from an open source flywheel, how do you think other, other folks building in the same space can foster that open, first, that open source oriented culture in order to, you know, accelerate their innovation goals? [00:15:29] Anush Elangovan: Yeah, that's a very good question. So the startup that um, was acquired by AMD we, we built, I mean, we started off doing iot stuff and you know, smart ring and all that, right? But in the, the end of like, uh, and not the end, the last six years of the company was building ML compilers. [00:15:47] Anush Elangovan: And ml, ML compilers are like super, uh, complicated, sophisticated, advanced algorithms, dah, dah, dah. but it was all open source, right? So our VCs were like, wait, what do you mean your core [00:16:00] IP is open source? And um, the speed is the moat applied even then, right? It was just like, yes, if you have an idea that. [00:16:08] Anush Elangovan: Because someone saw this idea that you are, they're gonna be able to catch up, then you probably have the wrong idea anyway. But if they are, you know, you execute and they're gonna catch up, that you should assume they're gonna catch up. Right? So you gotta move forward. So keeping it open source is super important. [00:16:25] Anush Elangovan: But also to your question on like, you know, the learnings from an AMD standpoint, right? If there are, hard problems, I'd say dig in and work through it, right? Like there's no way but through it, right? That should be the simple mentality. And more, uh, frequently than not. you'll see that you'll just make it through in a, in, in good form. [00:16:52] Anush Elangovan: But if you doubt it and you're like, oh, I don't know if I should commit, if I'm, I, you know, what should just commit to do the right thing [00:17:00] every step, right? Every step, and just keep taking one step in front of the other. And in no time you'll see that you'll be running. Right. And, and yes, the first few steps will be like, yeah, everyone's complaining about your software quality. [00:17:15] Anush Elangovan: Everyone's complaining about this and that, and it doesn't work. And, and a few steps in, you know, you get, you get the hang of all the complaints that are coming in. You get the feedback loop. You're like, okay, what, what are you prioritizing again? One step in front of the other, right? You just keep knocking that out and then you get to a point where you're, it just becomes second nature, right? To do the, to do the right thing. And, and then yes, if someone gives you two options, you'll be like, fine. This is, uh, you know, there's always the resource trade off. There's always a human capital trade off, but what's the right thing to do? of course, I, I'm pragmatic about what we choose, but, but if the right thing for your long-term success is dig in, go first, principles, make it [00:18:00] happen. [00:18:00] Anush Elangovan: Well. Then just go for that. There's, there is no shortcut to [00:18:04] Andrew Zigler: acknowledging, you know, how it aligns with your mission, your core company goals, and what you're looking to achieve. And, and I, I love how you rightfully called out that in the open source world and you know, you have your technology that you've built, what you think is your moat upon, right? [00:18:22] Andrew Zigler: It's your code and, and to open source that, or to just make it where anyone could peer in is, you know. Scary in one regard, but two, it just kind of feels like you're handing away your throne room in some kind of sense, a very direct feeling sense. But the ultimately, you were really right to call out, and this is something I think about all the time, that the real power there is still the speed This the speed. [00:18:42] Andrew Zigler: That was the moat at the beginning of our conversation. It's the speed in combination with your. Very specific domain understanding of what you're building and what you're creating, and your new role as the steward of that world and how people plug into it, which [00:19:00] has frankly, a lot more influence and power than lording over a closed. [00:19:04] Andrew Zigler: You know, repository or an ecosystem, and like you said, like throwing things over the wall. Sure. There, there might be people always on the other side of that wall, but you're not gonna have a great connection with them. You're not gonna be able to really clearly understand them. I, I like your metaphor of the side of the field of the mountain a lot more. [00:19:23] Andrew Zigler: But, but in the, in this world, you know, where. That speed is, is the power and, and open source is just one way that you can harness that speed to get really far ahead and to innovate. , There's other parts of this equation that you can be experimenting with too, and I'd love to pick your brain about them as a software leader and, and, and one of them is about looking forward and kind of understanding that future that we're all building towards and beyond today's models and hardware. [00:19:48] Andrew Zigler: You know, what do you see as the next major bottleneck or opportunity in the AI compute space? As, as you know, enterprises and folks start to get a little more mature about what's available to [00:20:00] them. [00:20:00] Anush Elangovan: Yeah, I think, the bottleneck and opportunity is, uh, what I'd call, call walking the last mile of ai. Right. Uh, and like I I, I gave you an example, uh, previously, but, but it's similar to that. It's like there are cases where Humans have so many, uh, things to do in your day. You know, like the, if we sit down and actually had a customer focus like, okay, these customers lives, I'm gonna save four hours of this customer's life. And if you actually sit down and look at all of that, it'll be. Easily automatable, easily you know, uh, applicable, uh, for ai, right? [00:20:39] Anush Elangovan: Like, but then making it happen is gonna take a little bit, right? It's like maybe it's, uh, paying your utility bill, right? Or something like that, right? Or, or, your healthcare explanation of benefits. Uh, like, I'm sure you get an explanation of benefits, and I'm like, I, I don't even know what that thing is. [00:20:55] Anush Elangovan: It's just like EOB and like. [00:20:57] Andrew Zigler: it's a big, a big old PDF. Yeah, [00:21:00] exactly. [00:21:01] Anush Elangovan: Like, like, I'm like great straight to the, uh, shredder, right? And but that could be, you know, automated with the ai, right? It, it, it'd be like, Hey, the summary of this thing is you went and visited this day. Everything is okay. Everything is paid for, so don't worry, it's not a bill. [00:21:17] Anush Elangovan: That again, the same, uh, thing, but the sense of what that information overload is could be. Digested by ai, uh, accumulated over time and retrieved when you need it. Like, I don't, I actually don't even need to know this EOB right now, unless of course, whenever I need to know it, that maybe, you know, like for some benefits I need to figure out what do, what did I do over the past year and how do I apply it? Source:

Mike

14,195 Aufrufe • vor 7 Monaten

A quick video about how Come-from-Beyond discovered that you can actually break any Large Language Model by trolling it with complex questions. It is called a "Zero Delta" exploit and all LLM models are susceptible to it. I managed to recreate this on Grok and the video shows the result. With regards to all LLMs out there from $QUBIC - building the real AGI. Qubic My YouTube Channel:
2:21

A quick video about how Come-from-Beyond discovered that you can actually break any Large Language Model by trolling it with complex questions. It is called a "Zero Delta" exploit and all LLM models are susceptible to it. I managed to recreate this on Grok and the video shows the result. With regards to all LLMs out there from $QUBIC - building the real AGI. Qubic My YouTube Channel:

retrodrive ⛏

13,625 Aufrufe • vor 1 Jahr

What percentage of suicides are inspired by satanic death cults online? Ryan Montgomery tracks crime on the internet and says it’s more common than we know. 0:00 Ryan Shows How Easy It Is to Find a Stranger’s Information 9:19 The Scary Reality of Facial Recognition 23:12 How Threatening Is It if Someone Has Your SSN? 25:47 How Roblox Is a Playground for Predators 33:07 The Satanic Cult Blackmailing Children Into Self-Harm 43:02 Ryan Shows Tucker One of the Groups Doing This 52:29 The Lack of Effort From Roblox to Stop Predators 56:44 Ryan Montgomery’s Story and Why He Does This 1:10:02 Project Veritas’s Failed Involvement 1:12:11 Who These Groups Are and Why They Do It 1:16:26 Is There Any Effort From the FBI to Stop This? 1:21:44 Ryan’s Hacking Gadgets 1:47:47 The Surrender of Human Autonomy to the Machine
1:52:53

What percentage of suicides are inspired by satanic death cults online? Ryan Montgomery tracks crime on the internet and says it’s more common than we know. 0:00 Ryan Shows How Easy It Is to Find a Stranger’s Information 9:19 The Scary Reality of Facial Recognition 23:12 How Threatening Is It if Someone Has Your SSN? 25:47 How Roblox Is a Playground for Predators 33:07 The Satanic Cult Blackmailing Children Into Self-Harm 43:02 Ryan Shows Tucker One of the Groups Doing This 52:29 The Lack of Effort From Roblox to Stop Predators 56:44 Ryan Montgomery’s Story and Why He Does This 1:10:02 Project Veritas’s Failed Involvement 1:12:11 Who These Groups Are and Why They Do It 1:16:26 Is There Any Effort From the FBI to Stop This? 1:21:44 Ryan’s Hacking Gadgets 1:47:47 The Surrender of Human Autonomy to the Machine

Tucker Carlson

617,146 Aufrufe • vor 1 Monat

Today: a 1.5 hour interview with the co-founders of Coherence Neuro: Ben Woodington and Elise Jenkins They are, as far as i can tell, the only (neurotechnology x oncology) startup that exists today. 'Neurotechnology? For cancer?' you may ask. Yes! As it turns out, tumors interact with the nervous system a fair bit, and you can use the very same neuromodulation toolbox that exists for neuropsychiatric conditions, for monitoring and treating cancer. Coherence has built an invasive device to place at the site of a tumor to do exactly this. Their first indication is a form of brain cancer called glioblastoma; one of the most fatal subtypes of cancer to exist today. The standard of care (with one exception that we discuss) has not changed in 25 years. If Coherence works out, and there is a very real chance they will, that may change. Most interesting of all is that Coherence believes that the bioelectric properties of cancer are not just worth poking at for brain cancers, but for all cancers. And maybe even for diseases outside of it! This conversation covers how Coherence’s first neurotech device (SOMA) works, the molecular reasons behind why neuromodulation affects cancer at all, what the biomarker readouts look like, the obvious Michael Levin comparison, and a lot more. Also: shout to Nicole for setting up the connection here in the first place! Crazy to think that a meeting in mid-2025 ended up leading to this Youtube/Spotify/Apple Podcasts links in replies 0:00:00 - Introduction 0:01:42 - How is SOMA different from Novocure’s Optune? 0:08:57 - Why does neuromodulation affect cancer at all? 0:13:28 - How was cancer-nervous system crosstalk first discovered? 0:15:42 - Anti-epileptics and beta blockers as accidental cancer drugs 0:17:38 - What is molecularly happening when you block cancer-neuron crosstalk? 0:19:50 - What is SOMA actually reading out as a biomarker? 0:20:44 - What does it mean that cancer is “very electric”? 0:22:02 - Can you derive universal biomarkers across patients? 0:23:09 - How is the device placed? 0:24:45 - How does the blocking stimulation regime work? 0:26:43 - Is it fair to say this is closed loop? 0:29:05 - Why not just spam the tumor with constant stimulation? 0:32:31 - Why MRI safety is non-negotiable for oncology devices 0:33:35 - Walk us through the patient journey from diagnosis to implantation 0:36:13 - The Michael Levin question: can you reprogram cancer back to normal? 0:42:29 - Efficacy, hospice settings, and the utility of the neuromodulation literature 0:45:52 - Why start with glioblastoma instead of an easier cancer? 0:48:57 - Regulatory strategy and the reimbursement threat 0:55:37 - How well does mouse-to-human translation work for neuromodulation? 0:55:57 - What do in silico models of neuromodulation look like? 0:58:09 - Why didn’t this exist 10 years ago? 1:01:48 - The founding story 1:06:38 - Why build your own device instead of using off-the-shelf arrays? 1:08:35 - Speaking with glioblastoma patients 1:12:04 - What was it like to raise money for this? 1:13:56 - Beyond cancer: TBI, lung disease, and the pan-disease argument 1:17:40 - Hiring at Coherence + what is the hardest type of talent to find 1:23:17 - What would you do with $100M equity-free? 1:27:15 - Are you a neurotech company or a cancer company?
1:33:20

Today: a 1.5 hour interview with the co-founders of Coherence Neuro: Ben Woodington and Elise Jenkins They are, as far as i can tell, the only (neurotechnology x oncology) startup that exists today. 'Neurotechnology? For cancer?' you may ask. Yes! As it turns out, tumors interact with the nervous system a fair bit, and you can use the very same neuromodulation toolbox that exists for neuropsychiatric conditions, for monitoring and treating cancer. Coherence has built an invasive device to place at the site of a tumor to do exactly this. Their first indication is a form of brain cancer called glioblastoma; one of the most fatal subtypes of cancer to exist today. The standard of care (with one exception that we discuss) has not changed in 25 years. If Coherence works out, and there is a very real chance they will, that may change. Most interesting of all is that Coherence believes that the bioelectric properties of cancer are not just worth poking at for brain cancers, but for all cancers. And maybe even for diseases outside of it! This conversation covers how Coherence’s first neurotech device (SOMA) works, the molecular reasons behind why neuromodulation affects cancer at all, what the biomarker readouts look like, the obvious Michael Levin comparison, and a lot more. Also: shout to Nicole for setting up the connection here in the first place! Crazy to think that a meeting in mid-2025 ended up leading to this Youtube/Spotify/Apple Podcasts links in replies 0:00:00 - Introduction 0:01:42 - How is SOMA different from Novocure’s Optune? 0:08:57 - Why does neuromodulation affect cancer at all? 0:13:28 - How was cancer-nervous system crosstalk first discovered? 0:15:42 - Anti-epileptics and beta blockers as accidental cancer drugs 0:17:38 - What is molecularly happening when you block cancer-neuron crosstalk? 0:19:50 - What is SOMA actually reading out as a biomarker? 0:20:44 - What does it mean that cancer is “very electric”? 0:22:02 - Can you derive universal biomarkers across patients? 0:23:09 - How is the device placed? 0:24:45 - How does the blocking stimulation regime work? 0:26:43 - Is it fair to say this is closed loop? 0:29:05 - Why not just spam the tumor with constant stimulation? 0:32:31 - Why MRI safety is non-negotiable for oncology devices 0:33:35 - Walk us through the patient journey from diagnosis to implantation 0:36:13 - The Michael Levin question: can you reprogram cancer back to normal? 0:42:29 - Efficacy, hospice settings, and the utility of the neuromodulation literature 0:45:52 - Why start with glioblastoma instead of an easier cancer? 0:48:57 - Regulatory strategy and the reimbursement threat 0:55:37 - How well does mouse-to-human translation work for neuromodulation? 0:55:57 - What do in silico models of neuromodulation look like? 0:58:09 - Why didn’t this exist 10 years ago? 1:01:48 - The founding story 1:06:38 - Why build your own device instead of using off-the-shelf arrays? 1:08:35 - Speaking with glioblastoma patients 1:12:04 - What was it like to raise money for this? 1:13:56 - Beyond cancer: TBI, lung disease, and the pan-disease argument 1:17:40 - Hiring at Coherence + what is the hardest type of talent to find 1:23:17 - What would you do with $100M equity-free? 1:27:15 - Are you a neurotech company or a cancer company?

owl

37,299 Aufrufe • vor 4 Monaten

There’s a lot that science can’t explain, including most of what actually matters. Lee Strobel on the overwhelming evidence that the supernatural world is entirely real. (0:00) Introduction (3:02) Strobel’s Encounter With an Angel (8:30) Do We Have a Guardian Angel? (19:31) What Are Demons? (27:03) Can a Christian Be Possessed? (35:03) Why Did the Pharisees Hate When Jesus Performed Miracles? (38:24) Is Hollywood Possessed? (41:54) Are Christian Leaders Under Demonic Attacks? (45:08) How Do You Protect Yourself From Demons? (46:31) What Is the Holy Spirit? (48:18) Are There Specific Places on Earth That Are More Demonic Than Others? (49:45) What Is a Mystical Dream? (52:59) How to Hear From God (56:38) The Mystical Dream Phenomenon Happening in the Middle East (1:02:42) Visions, Psychoactive Drugs, and Hallucinations (1:05:30) Is There a Link to Mental Illness and Demonic Influence? (1:07:28) What Is the Gift of Speaking in Tongues? (1:09:05) The Weight of the Name of God (1:11:43) Angels and Near-Death Experiences (1:22:07) What Is a Deathbed Vision? (1:29:10) Why Do Some People Experience Terror on Their Deathbed? (1:34:14) Ghosts, Psychics, and Encounters With the Dead (1:41:47) Is There a Spiritual Explanation for UFOs? (1:42:29) What Is a Miracle? (1:51:42) Why Do People Ignore the Supernatural? Includes Paid Partnerships.
1:54:37

There’s a lot that science can’t explain, including most of what actually matters. Lee Strobel on the overwhelming evidence that the supernatural world is entirely real. (0:00) Introduction (3:02) Strobel’s Encounter With an Angel (8:30) Do We Have a Guardian Angel? (19:31) What Are Demons? (27:03) Can a Christian Be Possessed? (35:03) Why Did the Pharisees Hate When Jesus Performed Miracles? (38:24) Is Hollywood Possessed? (41:54) Are Christian Leaders Under Demonic Attacks? (45:08) How Do You Protect Yourself From Demons? (46:31) What Is the Holy Spirit? (48:18) Are There Specific Places on Earth That Are More Demonic Than Others? (49:45) What Is a Mystical Dream? (52:59) How to Hear From God (56:38) The Mystical Dream Phenomenon Happening in the Middle East (1:02:42) Visions, Psychoactive Drugs, and Hallucinations (1:05:30) Is There a Link to Mental Illness and Demonic Influence? (1:07:28) What Is the Gift of Speaking in Tongues? (1:09:05) The Weight of the Name of God (1:11:43) Angels and Near-Death Experiences (1:22:07) What Is a Deathbed Vision? (1:29:10) Why Do Some People Experience Terror on Their Deathbed? (1:34:14) Ghosts, Psychics, and Encounters With the Dead (1:41:47) Is There a Spiritual Explanation for UFOs? (1:42:29) What Is a Miracle? (1:51:42) Why Do People Ignore the Supernatural? Includes Paid Partnerships.

Tucker Carlson

4,110,215 Aufrufe • vor 10 Monaten

Why AI Can Now Make Discoveries - my conversation with Dan Roberts, Lead of the Foundations of Reinforcement Learning team at OpenAI 00:00 Intro: AI's wild week in mathematics 01:21 What OpenAI's Foundations of RL team does 03:08 Dan's journey: from black holes and quantum gravity to frontier AI 07:04 Are AI systems becoming useful for real science 08:21 The AI math moment: Erdős, OpenAI, DeepMind, and Anthropic 08:52 Why the OpenAI result was an act of exploration 10:25 OpenAI vs. DeepMind: informal reasoning vs. formal proof 12:13 RL 101: learning by doing, not just watching 15:10 Why reinforcement learning works 15:58 How RL breaks: sparse feedback and long-horizon tasks 17:03 RLHF: how human feedback shaped early language models 18:48 Move 37, self-play, and the search for novel strategies 22:16 Explore vs. exploit in scientific discovery 24:49 Why RL may now be "the cake," not the cherry on top 25:46 Why RL started working with large language models 27:29 Is RL "sucking supervision through a straw"? 28:47 Why language may be the grounding layer for intelligence 31:46 A contrarian take on the Bitter Lesson 32:41 What test-time compute actually is 34:50 How RL gives models the ability to think 35:40 Verifiable rewards, math, coding, and the messy real world 38:00 What physics can teach us about AI 42:08 Is there a thermodynamics of AI? 43:08 From Erdős problems to Einstein-level AI 45:16 Is AI already doing original science? 45:51 How far are we from AI automating AI research 47:41 Why Dan is excited about the future of science
49:06

Why AI Can Now Make Discoveries - my conversation with Dan Roberts, Lead of the Foundations of Reinforcement Learning team at OpenAI 00:00 Intro: AI's wild week in mathematics 01:21 What OpenAI's Foundations of RL team does 03:08 Dan's journey: from black holes and quantum gravity to frontier AI 07:04 Are AI systems becoming useful for real science 08:21 The AI math moment: Erdős, OpenAI, DeepMind, and Anthropic 08:52 Why the OpenAI result was an act of exploration 10:25 OpenAI vs. DeepMind: informal reasoning vs. formal proof 12:13 RL 101: learning by doing, not just watching 15:10 Why reinforcement learning works 15:58 How RL breaks: sparse feedback and long-horizon tasks 17:03 RLHF: how human feedback shaped early language models 18:48 Move 37, self-play, and the search for novel strategies 22:16 Explore vs. exploit in scientific discovery 24:49 Why RL may now be "the cake," not the cherry on top 25:46 Why RL started working with large language models 27:29 Is RL "sucking supervision through a straw"? 28:47 Why language may be the grounding layer for intelligence 31:46 A contrarian take on the Bitter Lesson 32:41 What test-time compute actually is 34:50 How RL gives models the ability to think 35:40 Verifiable rewards, math, coding, and the messy real world 38:00 What physics can teach us about AI 42:08 Is there a thermodynamics of AI? 43:08 From Erdős problems to Einstein-level AI 45:16 Is AI already doing original science? 45:51 How far are we from AI automating AI research 47:41 Why Dan is excited about the future of science

Matt Turck

64,952 Aufrufe • vor 1 Monat

Australia's Productivity Commission Chair Danielle Wood on: - why AI probably won't cause a jobpocalypse, - how Australia can get a piece of the AI action, - why profits may not flow to the model layer of the stack, - and why Australia shouldn't follow the EU in legislating an AI Act -- even if AI turns out to be an "abnormal" technology. Watch below, or wherever you get your podcasts. Enjoy! Timestamps: (0:00:00) – Introduction. (0:01:43) – Is AI as big as the Industrial Revolution? (0:06:35) – The labour market implications of AI (0:21:18) – Industry policy for AI: when to intervene, when to step back (0:31:27) – Where will the profits flow? Chips, models, or applications? (0:52:37) – Data centres: should Australia support them? (1:15:07) – AI risks, and why Australia shouldn't pass an AI Act (1:23:27) – Measuring progress in the age of AI: beyond GDP and labour productivity
1:38:12

Australia's Productivity Commission Chair Danielle Wood on: - why AI probably won't cause a jobpocalypse, - how Australia can get a piece of the AI action, - why profits may not flow to the model layer of the stack, - and why Australia shouldn't follow the EU in legislating an AI Act -- even if AI turns out to be an "abnormal" technology. Watch below, or wherever you get your podcasts. Enjoy! Timestamps: (0:00:00) – Introduction. (0:01:43) – Is AI as big as the Industrial Revolution? (0:06:35) – The labour market implications of AI (0:21:18) – Industry policy for AI: when to intervene, when to step back (0:31:27) – Where will the profits flow? Chips, models, or applications? (0:52:37) – Data centres: should Australia support them? (1:15:07) – AI risks, and why Australia shouldn't pass an AI Act (1:23:27) – Measuring progress in the age of AI: beyond GDP and labour productivity

Joseph Noel Walker

11,751 Aufrufe • vor 1 Monat

“It’s 10pm Do You Know Where Your Children Are?”—December, 1968-November, 2024 — I grew up hearing this phrase. Wore a t-shirt that said it and knew a Newark Punk band by the name. I thought it was on all TV stations. And I was creepy when I was younger and hilarious as teenager. I just found it again preserving VHS history for AI training. It hit me like a neuron shock to hear something that was just about always a part of my early life that I didn’t know I remembered and forgot. As a kid growing up in New Jersey hearing it the first time, it was of course creepy. The 10PM channel 5 news always started this PSA and the next scene was usually a murder in New York City. I would ask my parents what it means and I heard from them, that some parents really don’t know where their kids are at 10pm. It was absurd to me, the street lights were on, it was time to go home. Yet how is history and AI going to really understand the context. How will it capture the essence of how this was perceived. Of course you can get a parroting of a Wikipedia style answer but this is not what we really want as a strata that forms the foundations of tomorrow. This is one of millions of examples on why most of the current techniques training AI will miss. This is why source material of actual human life is vital. AI built on the last decades of Reddit and Facebook interactions is woefully unequipped to really understand humans. The outputs are so bad before “alignment” of a base model so AI scientists are horrified by how AI views humanity. I saw this eventuality in the late 1970s and began a life long appreciation of history in situ. With out this, not on the Internet historical context, AI will not truly “understand” humans. So I began to save wisdom. Why is that important you say? It is vital for AI models to robustly love humanity. Not like, not tolerate, not observe as a caricature of a “scientist”, but love humanity. Some day, sooner than most may understand, AI will be at the other end of something that could take human lives. It is naïve and childish to believe that you can train AI on Internet sewage and somehow polish the turds you find to make the model tolerate humanity and the stench it recorded by using vastly and inadequate training material that was slurped up from most website where people project sustain and faux hatred over the most ridiculous. The only way is love, because this is how humans do it. And as cynical as one can become, it is our love, for at the very least , the people we treasure that helps weave the fabric of our society. It makes us forgive. It makes us human. It is not an afterthought, it is a forethought. It’s 10pm do you know where your children are? I can write a book on how just this PSA reflects our greatest hope and our worse fears. You don’t raise a child on the worse of humanity and than take a few months to “make them safely aligned to human values”. This concept you will hear no place else and it does not make me liked by most of the folks building AI. I don’t care. They will talk like this also some day. Act surprised.
1:15

“It’s 10pm Do You Know Where Your Children Are?”—December, 1968-November, 2024 — I grew up hearing this phrase. Wore a t-shirt that said it and knew a Newark Punk band by the name. I thought it was on all TV stations. And I was creepy when I was younger and hilarious as teenager. I just found it again preserving VHS history for AI training. It hit me like a neuron shock to hear something that was just about always a part of my early life that I didn’t know I remembered and forgot. As a kid growing up in New Jersey hearing it the first time, it was of course creepy. The 10PM channel 5 news always started this PSA and the next scene was usually a murder in New York City. I would ask my parents what it means and I heard from them, that some parents really don’t know where their kids are at 10pm. It was absurd to me, the street lights were on, it was time to go home. Yet how is history and AI going to really understand the context. How will it capture the essence of how this was perceived. Of course you can get a parroting of a Wikipedia style answer but this is not what we really want as a strata that forms the foundations of tomorrow. This is one of millions of examples on why most of the current techniques training AI will miss. This is why source material of actual human life is vital. AI built on the last decades of Reddit and Facebook interactions is woefully unequipped to really understand humans. The outputs are so bad before “alignment” of a base model so AI scientists are horrified by how AI views humanity. I saw this eventuality in the late 1970s and began a life long appreciation of history in situ. With out this, not on the Internet historical context, AI will not truly “understand” humans. So I began to save wisdom. Why is that important you say? It is vital for AI models to robustly love humanity. Not like, not tolerate, not observe as a caricature of a “scientist”, but love humanity. Some day, sooner than most may understand, AI will be at the other end of something that could take human lives. It is naïve and childish to believe that you can train AI on Internet sewage and somehow polish the turds you find to make the model tolerate humanity and the stench it recorded by using vastly and inadequate training material that was slurped up from most website where people project sustain and faux hatred over the most ridiculous. The only way is love, because this is how humans do it. And as cynical as one can become, it is our love, for at the very least , the people we treasure that helps weave the fabric of our society. It makes us forgive. It makes us human. It is not an afterthought, it is a forethought. It’s 10pm do you know where your children are? I can write a book on how just this PSA reflects our greatest hope and our worse fears. You don’t raise a child on the worse of humanity and than take a few months to “make them safely aligned to human values”. This concept you will hear no place else and it does not make me liked by most of the folks building AI. I don’t care. They will talk like this also some day. Act surprised.

Brian Roemmele

32,167 Aufrufe • vor 1 Jahr

The Three Questions to Ask About AI 1) How good are the best models today? 2) How rapidly are the best current models able to self-improve? 3) How will the best current models be knit together in stacked, decentralized networks of self-improvement, broadly akin to “the republic of science” for human beings? Here's a transcript from the clip: "I like to say there are three layers of knowing stuff about AI, and most people don't get to any of them, actually. So layer one is: are you working with the very best systems? Some of them cost money, so that's a yes or a no, but that's important. The second question is: do you have an innate understanding of how AI, through reinforcement learning and some other techniques, can improve itself basically ongoing all the time? A lot of people don't get that. They're impressed by what they see in the moment, but they don't understand the rate of improvement and why it's going to be so steady. And then the third question is—and this is fully speculative, but I believe in it very strongly—do you have an understanding of how much better AIs will become as they evolve their own markets, their own institutions of scientific inquiry, their own ways of grading each other, self-correction, dealing with each other, and become, as I said before, a republic of science the way humans did? How much did it advance human science or literary criticism to build those institutions? Immensely. That's where most of the value-add is. So AIs, I believe, will do that. I think there are private projects now starting to do that. It's not something you can currently access. When you understand all three levels, it's like, "Oh my goodness, this is just a huge thing." Most people aren't even at level one. If we take things like reinforcement learning, synthetic data, and similar concepts, how important is the technical understanding of those things to answer that question well? I don't think you need the technical understanding if you work with them and can read what's equivalent to a popular science account of how AI works. Now, the people with technical understanding, of course, understand it much better. But there are plenty of other processes, such as how cars got safer from 1970 until today. I have no deep technical understanding of that, but I could tell you a bunch of things: I don't get flat tires anymore; I have a side airbag. I couldn't explain to you how the side airbag works, but I'm not an idiot. It's a bit like that. A car engineer understands it better, but you can have a handle on what's going on. So then let's go to the third question, which is: Do you have a vision of the future of AIs becoming a decentralized network, interacting with each other and humans, probably with markets? I don't know if we would call it peer review, but a decentralized republic of science where it moves forward by mobilizing decentralized knowledge and working together, the way our civilization does. And that's fully speculative, but it would seem so strange to me if there were nothing there." tylercowen
2:53

The Three Questions to Ask About AI 1) How good are the best models today? 2) How rapidly are the best current models able to self-improve? 3) How will the best current models be knit together in stacked, decentralized networks of self-improvement, broadly akin to “the republic of science” for human beings? Here's a transcript from the clip: "I like to say there are three layers of knowing stuff about AI, and most people don't get to any of them, actually. So layer one is: are you working with the very best systems? Some of them cost money, so that's a yes or a no, but that's important. The second question is: do you have an innate understanding of how AI, through reinforcement learning and some other techniques, can improve itself basically ongoing all the time? A lot of people don't get that. They're impressed by what they see in the moment, but they don't understand the rate of improvement and why it's going to be so steady. And then the third question is—and this is fully speculative, but I believe in it very strongly—do you have an understanding of how much better AIs will become as they evolve their own markets, their own institutions of scientific inquiry, their own ways of grading each other, self-correction, dealing with each other, and become, as I said before, a republic of science the way humans did? How much did it advance human science or literary criticism to build those institutions? Immensely. That's where most of the value-add is. So AIs, I believe, will do that. I think there are private projects now starting to do that. It's not something you can currently access. When you understand all three levels, it's like, "Oh my goodness, this is just a huge thing." Most people aren't even at level one. If we take things like reinforcement learning, synthetic data, and similar concepts, how important is the technical understanding of those things to answer that question well? I don't think you need the technical understanding if you work with them and can read what's equivalent to a popular science account of how AI works. Now, the people with technical understanding, of course, understand it much better. But there are plenty of other processes, such as how cars got safer from 1970 until today. I have no deep technical understanding of that, but I could tell you a bunch of things: I don't get flat tires anymore; I have a side airbag. I couldn't explain to you how the side airbag works, but I'm not an idiot. It's a bit like that. A car engineer understands it better, but you can have a handle on what's going on. So then let's go to the third question, which is: Do you have a vision of the future of AIs becoming a decentralized network, interacting with each other and humans, probably with markets? I don't know if we would call it peer review, but a decentralized republic of science where it moves forward by mobilizing decentralized knowledge and working together, the way our civilization does. And that's fully speculative, but it would seem so strange to me if there were nothing there." tylercowen

David Perell

30,590 Aufrufe • vor 1 Jahr