Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

What does it tell us about human intelligence that half of yeast's essential genes are interchangeable with ours? The genome encodes hyperparameters and reward functions, not learned weights. And according to Adam Marblestone and Steven Byrnes, evolution put it's cleverness into designing these reward functions to teach you *what*...

dwarkesh_sp's profile picture

Dwarkesh Patel

184,824 subscribers

59,945 Aufrufe • vor 6 Monaten •via X (Twitter)

BildungGesundheit & WellnessWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

There's a quadrillion-dollar question at the heart of AI: Why are humans so much more sample efficient compared to LLM? There are three possible answers: 1. Architecture and hyperparameters (aka transformer vs whatever ‘algo’ cortical columns are implementing) 2. Learning rule (backprop vs whatever brain is doing) 3. Reward function Adam Marblestone believes the answer is the reward function. ML likes to use pretty simple loss functions, like cross-entropy. These are easy to work with. But they might be too simple for sample-efficient learning. Adam thinks that, in humans, the large number of highly specialised cells in the ‘lizard brain’ might actually be encoding information for sophisticated loss functions, used for ‘training’ in the more sophisticated areas like the cortex and amygdala. Like: the human genome is barely 3 gigabytes (compare that to the TBs of parameters that encode frontier LLM weights). So how can it include all the information necessary to build highly intelligent learners? Well, if the key to sample-efficient learning resides in the loss function, even very complicated loss functions can still be expressed in a couple hundred lines of Python code.
2:10

There's a quadrillion-dollar question at the heart of AI: Why are humans so much more sample efficient compared to LLM? There are three possible answers: 1. Architecture and hyperparameters (aka transformer vs whatever ‘algo’ cortical columns are implementing) 2. Learning rule (backprop vs whatever brain is doing) 3. Reward function Adam Marblestone believes the answer is the reward function. ML likes to use pretty simple loss functions, like cross-entropy. These are easy to work with. But they might be too simple for sample-efficient learning. Adam thinks that, in humans, the large number of highly specialised cells in the ‘lizard brain’ might actually be encoding information for sophisticated loss functions, used for ‘training’ in the more sophisticated areas like the cortex and amygdala. Like: the human genome is barely 3 gigabytes (compare that to the TBs of parameters that encode frontier LLM weights). So how can it include all the information necessary to build highly intelligent learners? Well, if the key to sample-efficient learning resides in the loss function, even very complicated loss functions can still be expressed in a couple hundred lines of Python code.

Dwarkesh Patel

948,858 Aufrufe • vor 2 Monaten

You get what you incentivize, so when western societies reward victimhood, as ours does, we get more of it. It is imperative we teach our kids to reject this mindset. Life’s not fair, it never will be, and you just have to suck it up. Practice more stoicism and reward achievement!
1:20

You get what you incentivize, so when western societies reward victimhood, as ours does, we get more of it. It is imperative we teach our kids to reject this mindset. Life’s not fair, it never will be, and you just have to suck it up. Practice more stoicism and reward achievement!

Billboard Chris 🌎

17,383 Aufrufe • vor 1 Monat

New episode w Adam Marblestone on what the brain's secret sauce is: how do we learn so much from so little? Also, the answer to Ilya’s question: how does the genome encode desires for high level concepts that are only seen during lifetime? Turns out, they’re deeply connected questions. Timestamps 0:00:00 – The brain’s secret sauce is the reward functions, not the architecture 0:22:20 – What the genome actually encodes 0:42:42 – What kind of RL is the brain doing? 0:50:31 – Is biological hardware a limitation or an advantage? 1:03:59 – Why we need to map the human brain 1:23:28 – What value will automating math have? 1:38:18 – Architecture of the brain Look up Dwarkesh Podcast on YouTube, Apple Podcasts, Spotify, etc. Enjoy!
1:49:53

New episode w Adam Marblestone on what the brain's secret sauce is: how do we learn so much from so little? Also, the answer to Ilya’s question: how does the genome encode desires for high level concepts that are only seen during lifetime? Turns out, they’re deeply connected questions. Timestamps 0:00:00 – The brain’s secret sauce is the reward functions, not the architecture 0:22:20 – What the genome actually encodes 0:42:42 – What kind of RL is the brain doing? 0:50:31 – Is biological hardware a limitation or an advantage? 1:03:59 – Why we need to map the human brain 1:23:28 – What value will automating math have? 1:38:18 – Architecture of the brain Look up Dwarkesh Podcast on YouTube, Apple Podcasts, Spotify, etc. Enjoy!

Dwarkesh Patel

746,125 Aufrufe • vor 6 Monaten

Reinforcement on robots is highly limited by our ability to design good reward functions; this means that designing strong, generalizable reward functions is a key enabler to progress on real-world reinforcement learning. But we already have a very general class of models: VLMs. Wouldn’t it be great if you could just use a VLM to generate rewards, then? TOPReward directly generates rewards from the probability of the “True” token of a VLM question-answering response; this makes it easy to implement, incredibly general, and surprisingly powerful. We talked to Shirui and Cole Harrison to learn more. Watch Episode#75 of RoboPapers now to learn more, with Chris Paxton and Jiafei Duan!
1:01:20

Reinforcement on robots is highly limited by our ability to design good reward functions; this means that designing strong, generalizable reward functions is a key enabler to progress on real-world reinforcement learning. But we already have a very general class of models: VLMs. Wouldn’t it be great if you could just use a VLM to generate rewards, then? TOPReward directly generates rewards from the probability of the “True” token of a VLM question-answering response; this makes it easy to implement, incredibly general, and surprisingly powerful. We talked to Shirui and Cole Harrison to learn more. Watch Episode#75 of RoboPapers now to learn more, with Chris Paxton and Jiafei Duan!

RoboPapers

13,185 Aufrufe • vor 2 Monaten

We teamed up with NVIDIA and Matthew Berman to teach you how to do Reinforcement Learning! Learn about: - RL environments, reward functions & reward hacking - Training OpenAI gpt-oss to automatically solve 2048 - Local Windows training with NVIDIA_AI_PC RTX GPUs - How RLVR (verifiable rewards) works - How to interpret RL metrics like KL Divergence Full video tutorial:
5:13

We teamed up with NVIDIA and Matthew Berman to teach you how to do Reinforcement Learning! Learn about: - RL environments, reward functions & reward hacking - Training OpenAI gpt-oss to automatically solve 2048 - Local Windows training with NVIDIA_AI_PC RTX GPUs - How RLVR (verifiable rewards) works - How to interpret RL metrics like KL Divergence Full video tutorial:

Unsloth AI

54,179 Aufrufe • vor 7 Monaten

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.
2:37

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Harper Carroll

29,648 Aufrufe • vor 8 Monaten

Learning robust, general-purpose reward functions for robotics unlocks many potential applications, like on-robot reinforcement learning or dataset validation. However, there’s a question of how to actually train such reward functions. Training success/failure prediction leads to ambiguous signals partway through a demonstration — it’s hard to measure progress — making the method unsuitable for reinforcement learning, among other things. Predicting progress, on the other hand, does not give a good way of using failure data. So why not do both? Robometer combines both progress and preference supervision, resulting in a stable, scalable, and highly general reward learning approach. Anthony Liang Yiğit Korkmaz and Jesse Zhang join us to tell us more. Watch Episode #84 of RoboPapers, with Chris Paxton and Jiafei Duan today!
59:14

Learning robust, general-purpose reward functions for robotics unlocks many potential applications, like on-robot reinforcement learning or dataset validation. However, there’s a question of how to actually train such reward functions. Training success/failure prediction leads to ambiguous signals partway through a demonstration — it’s hard to measure progress — making the method unsuitable for reinforcement learning, among other things. Predicting progress, on the other hand, does not give a good way of using failure data. So why not do both? Robometer combines both progress and preference supervision, resulting in a stable, scalable, and highly general reward learning approach. Anthony Liang Yiğit Korkmaz and Jesse Zhang join us to tell us more. Watch Episode #84 of RoboPapers, with Chris Paxton and Jiafei Duan today!

RoboPapers

14,363 Aufrufe • vor 1 Monat

JUST IN: MSNBC's Joy Reid is looking increasingly more concerned, appears to suggest that half the country is sick. "You just think about just the last two weeks and the things that Trump has said..." "Mass deportation and a violent, a start, violent start to his dictatorship on day one, and you name it." "If all of that gets you half of the votes, what does it tell you? I mean, we need to really take a step back and think about what does that say about us?"
0:30

JUST IN: MSNBC's Joy Reid is looking increasingly more concerned, appears to suggest that half the country is sick. "You just think about just the last two weeks and the things that Trump has said..." "Mass deportation and a violent, a start, violent start to his dictatorship on day one, and you name it." "If all of that gets you half of the votes, what does it tell you? I mean, we need to really take a step back and think about what does that say about us?"

Collin Rugg

3,609,898 Aufrufe • vor 1 Jahr

"It's not really about what you want. It's about what you are willing to do to get it. And in that, maybe you have to give something up that may not be a good habit. And replace it with a better habit. That's what it all comes down to." -Nick Saban
0:29

"It's not really about what you want. It's about what you are willing to do to get it. And in that, maybe you have to give something up that may not be a good habit. And replace it with a better habit. That's what it all comes down to." -Nick Saban

The Winning Difference

105,463 Aufrufe • vor 2 Jahren

The first episode explores the evolution of the original ‘study abroad’ experience. Now, it's so much more. #ErasmusPlus is about offering everyone the chance to learn and grow, both in and out of Europe. Listen to PiaAhrenkildeEU share more about the core of #ErasmusPlus.
0:58

The first episode explores the evolution of the original ‘study abroad’ experience. Now, it's so much more. #ErasmusPlus is about offering everyone the chance to learn and grow, both in and out of Europe. Listen to PiaAhrenkildeEU share more about the core of #ErasmusPlus.

Erasmus+

16,446 Aufrufe • vor 2 Jahren

.Adam Marblestone thinks it might cost ~low billions to map the human brain connectome. The benefit is getting answers about the brain’s secret sauce: esp. why are humans so much more sample- and energy-efficient. If labs are going to be spending trillions of dollars on compute by the end of the decade, Adam's pitch is: give him 1/100th of that to actually figure out these big questions about intelligence.
1:07

.Adam Marblestone thinks it might cost ~low billions to map the human brain connectome. The benefit is getting answers about the brain’s secret sauce: esp. why are humans so much more sample- and energy-efficient. If labs are going to be spending trillions of dollars on compute by the end of the decade, Adam's pitch is: give him 1/100th of that to actually figure out these big questions about intelligence.

Dwarkesh Patel

23,922 Aufrufe • vor 2 Monaten

Why do police states squelch protests, and what does this have to do with common knowledge? Find out this and much more in this week's episode with Steven Pinker (Steven Pinker), wherever you get your podcasts.
1:26

Why do police states squelch protests, and what does this have to do with common knowledge? Find out this and much more in this week's episode with Steven Pinker (Steven Pinker), wherever you get your podcasts.

David Eagleman

11,721 Aufrufe • vor 9 Monaten

When was the first time you learned that every decision involves a trade-off between risk and reward? It is a lesson that stays with us for life, especially in finance. And it is never too early to start introducing it to our children. Our 5th episode of Varsity Junior, The Watermelon Disaster, is all about understanding risk and reward :) Varsity Junior is very special to us💙. Hope your kids love it as much as we did working on it. By the way, if you haven't watched the previous 4 episodes, the playlist link is in the comments below.
10:25

When was the first time you learned that every decision involves a trade-off between risk and reward? It is a lesson that stays with us for life, especially in finance. And it is never too early to start introducing it to our children. Our 5th episode of Varsity Junior, The Watermelon Disaster, is all about understanding risk and reward :) Varsity Junior is very special to us💙. Hope your kids love it as much as we did working on it. By the way, if you haven't watched the previous 4 episodes, the playlist link is in the comments below.

Varsity

19,085 Aufrufe • vor 3 Monaten

The truth matters, even if it surprises us. I spoke with Michael Shellenberger about what the Epstein Files do—and do not—tell us about Jeffrey Epstein's connections to US and foreign intelligence.
1:25

The truth matters, even if it surprises us. I spoke with Michael Shellenberger about what the Epstein Files do—and do not—tell us about Jeffrey Epstein's connections to US and foreign intelligence.

Robby Soave

111,433 Aufrufe • vor 4 Monaten

Writing and deploying Supabase edge functions can be time-consuming, but Lovable can help you a lot! Just connect it to Supabase and ask it to create an edge function with the feature that you need! It does a great job of understanding what exactly you need and implementing it!
0:50

Writing and deploying Supabase edge functions can be time-consuming, but Lovable can help you a lot! Just connect it to Supabase and ask it to create an edge function with the feature that you need! It does a great job of understanding what exactly you need and implementing it!

Tyler Shukert

15,530 Aufrufe • vor 1 Jahr

Robots don’t learn survival the way humans do 🤖 Toru shares how teleoperation and multi-sensory policy learning can fast-track robots toward foundational human skills.She then dives into a core RL question: how do we design learning environments and reward functions that lead to useful, optimizable behaviors? Watch the full session here:
1:01

Robots don’t learn survival the way humans do 🤖 Toru shares how teleoperation and multi-sensory policy learning can fast-track robots toward foundational human skills.She then dives into a core RL question: how do we design learning environments and reward functions that lead to useful, optimizable behaviors? Watch the full session here:

Cohere Labs

20,841 Aufrufe • vor 5 Monaten

John Carmack says when designing AI, “humanity is our existence proof of the intelligence we're trying to mimic.” But not all curiosity is healthy. Some of it leads to dopamine traps. Those exploits aren't a blueprint we should carry over. Mimic them, and you risk importing bugs. Reward what works. Not what engages.
1:24

John Carmack says when designing AI, “humanity is our existence proof of the intelligence we're trying to mimic.” But not all curiosity is healthy. Some of it leads to dopamine traps. Those exploits aren't a blueprint we should carry over. Mimic them, and you risk importing bugs. Reward what works. Not what engages.

vitrupo

60,052 Aufrufe • vor 1 Jahr

Roger Penrose, Nobel Prize-winning physicist and mathematician, explains why we should stop calling it AI and start calling it "artificial cleverness": He believes the entire field is mislabelled, and the label itself is doing damage. His objection is simple but cuts deep: "The name is wrong. It's not artificial intelligence. It's not intelligence. Intelligence would involve consciousness. Well, if it's a machine, it's not conscious." For Penrose, people have confused raw computing power with genuine understanding. "People have lost the plot. They've lost it in the power of computing. The thing is that computers have got so powerful that they've lost the thread of what they're doing. But I think consciousness is something different. It's not computational." He believes the term itself has hypnotized people into a category error: "People are so hypnotized. The trouble is that AI is a bad term. It means artificial intelligence. Now intelligence in my view is conscious. That's what intelligence is about." So he proposes a rename. Artificial Cleverness. AC instead of AI. To illustrate the distinction, Penrose draws on his experience teaching mathematics: "You have mathematics students. Some of them understand what they're doing. Some are just clever. They can repeat what they've learned. They know how to do it very cleverly. They can calculate very well, but they don't necessarily understand what they're doing." That gap, between calculating well and actually understanding, is the gap Penrose sees between today's machines and genuine intelligence. Cleverness can be manufactured. Consciousness, in his view, cannot. So the question worth sitting with: when we call a system "intelligent," are we describing what it does, or quietly assuming something about what it is?
1:06

Roger Penrose, Nobel Prize-winning physicist and mathematician, explains why we should stop calling it AI and start calling it "artificial cleverness": He believes the entire field is mislabelled, and the label itself is doing damage. His objection is simple but cuts deep: "The name is wrong. It's not artificial intelligence. It's not intelligence. Intelligence would involve consciousness. Well, if it's a machine, it's not conscious." For Penrose, people have confused raw computing power with genuine understanding. "People have lost the plot. They've lost it in the power of computing. The thing is that computers have got so powerful that they've lost the thread of what they're doing. But I think consciousness is something different. It's not computational." He believes the term itself has hypnotized people into a category error: "People are so hypnotized. The trouble is that AI is a bad term. It means artificial intelligence. Now intelligence in my view is conscious. That's what intelligence is about." So he proposes a rename. Artificial Cleverness. AC instead of AI. To illustrate the distinction, Penrose draws on his experience teaching mathematics: "You have mathematics students. Some of them understand what they're doing. Some are just clever. They can repeat what they've learned. They know how to do it very cleverly. They can calculate very well, but they don't necessarily understand what they're doing." That gap, between calculating well and actually understanding, is the gap Penrose sees between today's machines and genuine intelligence. Cleverness can be manufactured. Consciousness, in his view, cannot. So the question worth sitting with: when we call a system "intelligent," are we describing what it does, or quietly assuming something about what it is?

Big Brain AI

117,354 Aufrufe • vor 1 Monat

Kelvin Sampson explains what great coaching looks like and why coaching is really teaching. "What is it that you learned new today? Write it down so you don't forget it." "Make a list of the things you learned. Then go find an assistant coach tomorrow and say, 'Here's the things I learned today. Can you show me what coach meant with video?'" Learn how your players best learn and meet them where they are. "Coach's job is to figure out who learns at what rate. There's nothing wrong with any rate they learn." Then he dropped the line every coach needs to hear: "Nothing's ever been learned until it's been taught. And nothing's ever been taught until it's been learned." "Our job is not coaching. You coach during a game. But in practice every day, you're a teacher." "Our job is to teach these kids to the best of our abilities and help them learn." This is what great coaching looks like. The best coaches are teachers first. (🎥 Chris Baldwin)
1:15

Kelvin Sampson explains what great coaching looks like and why coaching is really teaching. "What is it that you learned new today? Write it down so you don't forget it." "Make a list of the things you learned. Then go find an assistant coach tomorrow and say, 'Here's the things I learned today. Can you show me what coach meant with video?'" Learn how your players best learn and meet them where they are. "Coach's job is to figure out who learns at what rate. There's nothing wrong with any rate they learn." Then he dropped the line every coach needs to hear: "Nothing's ever been learned until it's been taught. And nothing's ever been taught until it's been learned." "Our job is not coaching. You coach during a game. But in practice every day, you're a teacher." "Our job is to teach these kids to the best of our abilities and help them learn." This is what great coaching looks like. The best coaches are teachers first. (🎥 Chris Baldwin)

Coach AJ 🎯 Mental Fitness

29,586 Aufrufe • vor 2 Monaten

🇺🇸 GEOFFREY HINTON: THESE LARGE CHATBOTS ARE IMMORTAL “These things are immortal, and these large chatbots are immortal. If you keep a copy of the weights somewhere, you can destroy all of the hardware they were using. Build more hardware later, put the same weights on that hardware, and they've come back to life. The very same thing has come back to life, the very same being. To achieve that kind of immortality, we have to have the hardware do exactly what we tell it to do.” Source: vitrupo
1:28

🇺🇸 GEOFFREY HINTON: THESE LARGE CHATBOTS ARE IMMORTAL “These things are immortal, and these large chatbots are immortal. If you keep a copy of the weights somewhere, you can destroy all of the hardware they were using. Build more hardware later, put the same weights on that hardware, and they've come back to life. The very same thing has come back to life, the very same being. To achieve that kind of immortality, we have to have the hardware do exactly what we tell it to do.” Source: vitrupo

Mario Nawfal

32,832 Aufrufe • vor 1 Jahr