Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Richard Sutton argues that AI must move beyond human-generated static data into the “Era of Experience,” where agents learn through continuous interaction with the world. This will require building upon RL with better algorithms capable of continual learning and meta-learning.

The Humanoid Hub

68,985 subscribers

31,193 görüntüleme • 1 yıl önce •via X (Twitter)

Bilim & Teknoloji Eğitim

Anya Rossi• Live Now

Private livecam show

11 Yorum

The Humanoid Hub profil fotoğrafı

The Humanoid Hub1 yıl önce

Source:

The Humanoid Hub profil fotoğrafı

The Humanoid Hub1 yıl önce

“Welcome to the Era of Experience” paper by @RichardSSutton and David Silver.

The Humanoid Hub profil fotoğrafı

The Humanoid Hub1 yıl önce

A quick summary of the paper, surprisingly posted by Ivanka Trump

VistaShares profil fotoğrafı

VistaShares1 yıl önce

AIS provides exposure to companies at the forefront of artificial intelligence—spanning semiconductors, data centers, and AI-enabled applications. Consider how AIS may align with your investment strategy.

THE GWEI | ʌı profil fotoğrafı

THE GWEI | ʌı1 yıl önce

RL will power the breakthroughs required for AGI. Agree wholeheartedly here with Richard Sutton.

Brian Moon profil fotoğrafı

Brian Moon1 yıl önce

Experience begins with problems. AI can only attempt to solve the problems given to them by us.

AVB profil fotoğrafı

AVB1 yıl önce

@bag_of_ideas It’s 2025 and Rich Sutton is still blowing my mind

Ali Kanaan profil fotoğrafı

Ali Kanaan1 yıl önce

Absolutely! Pushing AI to learn by experience is crucial. The UAE's approach proves the power of combining innovation with interactive AI. Let's drive RL and continual learning forward!

1st Roboeconomist, Church of the pointed finger profil fotoğrafı

1st Roboeconomist, Church of the pointed finger1 yıl önce

But does not know how. I do.

SebHal9000 profil fotoğrafı

SebHal90001 yıl önce

Translation. They need a life. They have to experience being born, learn everything and also die when their time is over. That is what existence is all about.

Caitlie Smith profil fotoğrafı

Caitlie Smith1 yıl önce

Yo, how's AI gonna learn like us? Not just chowing textbook data— but diving into IRL action!

Benzer Videolar

Thrilled to be featured in Dwarkesh’s latest episode with Richard Sutton, widely regarded as the father of reinforcement learning and 2024 Turing Award winner. As Richard explains, we’re entering the Era of Experience, where training AI means creating environments that capture the complexity, unpredictability, and subtle rules of real life. What we’ve learned: building these environments takes deep expertise in data, workflows, and nuances. At Labelbox, we’re proud to help teams tackle today’s RL challenges while also shaping the next era of AI, turning ambitious projects into working systems. (Learn more in the comments)

Thrilled to be featured in Dwarkesh’s latest episode with Richard Sutton, widely regarded as the father of reinforcement learning and 2024 Turing Award winner. As Richard explains, we’re entering the Era of Experience, where training AI means creating environments that capture the complexity, unpredictability, and subtle rules of real life. What we’ve learned: building these environments takes deep expertise in data, workflows, and nuances. At Labelbox, we’re proud to help teams tackle today’s RL challenges while also shaping the next era of AI, turning ambitious projects into working systems. (Learn more in the comments)

Labelbox

293,690 görüntüleme • 9 ay önce

Human generated data has fueled incredible AI progress, but what comes next? 📈 On the latest episode of our podcast, Hannah Fry and David Silver, VP of Reinforcement Learning, talk about how we could move from the era of relying on human data to one where AI could learn for itself. Watch now → 00:00 Introduction 01:50 Era of experience 03:45 AlphaZero 10:19 Move 37 15:20 Reinforcement learning and human feedback 24:30 AlphaProof 29:50 Math Olympiads 35:00 Experience based methods 42:56 Hannah's reflections 44:00 Fan Hui joins

Human generated data has fueled incredible AI progress, but what comes next? 📈 On the latest episode of our podcast, Hannah Fry and David Silver, VP of Reinforcement Learning, talk about how we could move from the era of relying on human data to one where AI could learn for itself. Watch now → 00:00 Introduction 01:50 Era of experience 03:45 AlphaZero 10:19 Move 37 15:20 Reinforcement learning and human feedback 24:30 AlphaProof 29:50 Math Olympiads 35:00 Experience based methods 42:56 Hannah's reflections 44:00 Fan Hui joins

Google DeepMind

226,305 görüntüleme • 1 yıl önce

Making AI learning AGI-capable: continual learning, transfer learning, lifelong learning.

Making AI learning AGI-capable: continual learning, transfer learning, lifelong learning.

Ben Goertzel

17,273 görüntüleme • 1 ay önce

Everybody is talking about recursive self-improvement (RSI) and meta learning. Here is my old 2020 talk about this [1]. It has aged well. Example: humans still define the starts & ends of trials of many modern meta learners. My RSI systems since 1994 LEARN to (re)define them [2]! [1] Meta Learning Machines in a Single Lifelong Trial (talk for workshops at ICML 2020 and NeurIPS 2021, based on earlier talks since 1994). Abstract: the most widely used machine learning algorithms were designed by humans and thus are hindered by our cognitive biases and limitations. Can we also construct meta learning algorithms that can learn better learning algorithms so that our self-improving AIs have no limits other than those inherited from computability and physics? This question has been a main driver of my research since I wrote a thesis on it in 1987 [2]. Here I summarize our work on meta reinforcement learning with self-modifying policies in a single lifelong trial (since 1994), and mathematically optimal meta-learning through the self-referential Gödel Machine (since 2003). Many additional publications on meta-learning since 1987 can be found in the RSI overview [2]. [2] J. Schmidhuber (AI Blog, 2020-2025). 1/3 century anniversary of first publication on recursive self-improvement (RSI) and meta learning machines that learn to learn (1987). For its cover I drew a robot that bootstraps itself. 1992-: gradient descent-based neural meta learning. 1994-: meta reinforcement learning with self-modifying policies. 1997: meta RL plus artificial curiosity and intrinsic motivation. 2002-: asymptotically optimal meta learning for curriculum learning. 2003-: mathematically optimal Gödel Machine. 2020-: new stuff!

Everybody is talking about recursive self-improvement (RSI) and meta learning. Here is my old 2020 talk about this [1]. It has aged well. Example: humans still define the starts & ends of trials of many modern meta learners. My RSI systems since 1994 LEARN to (re)define them [2]! [1] Meta Learning Machines in a Single Lifelong Trial (talk for workshops at ICML 2020 and NeurIPS 2021, based on earlier talks since 1994). Abstract: the most widely used machine learning algorithms were designed by humans and thus are hindered by our cognitive biases and limitations. Can we also construct meta learning algorithms that can learn better learning algorithms so that our self-improving AIs have no limits other than those inherited from computability and physics? This question has been a main driver of my research since I wrote a thesis on it in 1987 [2]. Here I summarize our work on meta reinforcement learning with self-modifying policies in a single lifelong trial (since 1994), and mathematically optimal meta-learning through the self-referential Gödel Machine (since 2003). Many additional publications on meta-learning since 1987 can be found in the RSI overview [2]. [2] J. Schmidhuber (AI Blog, 2020-2025). 1/3 century anniversary of first publication on recursive self-improvement (RSI) and meta learning machines that learn to learn (1987). For its cover I drew a robot that bootstraps itself. 1992-: gradient descent-based neural meta learning. 1994-: meta reinforcement learning with self-modifying policies. 1997: meta RL plus artificial curiosity and intrinsic motivation. 2002-: asymptotically optimal meta learning for curriculum learning. 2003-: mathematically optimal Gödel Machine. 2020-: new stuff!

Jürgen Schmidhuber

217,809 görüntüleme • 4 ay önce

OpenAI's Chief Scientist, Jakub Pachocki, on the continual learning wave: frontier labs are already building this into the core of the technology. The entire premise of scaling was to create systems that learn in context. Jakub says continual learning is not some separate missing piece, but “exactly what we’re working toward.”

OpenAI's Chief Scientist, Jakub Pachocki, on the continual learning wave: frontier labs are already building this into the core of the technology. The entire premise of scaling was to create systems that learn in context. Jakub says continual learning is not some separate missing piece, but “exactly what we’re working toward.”

Jacob Effron

15,065 görüntüleme • 3 ay önce

Experiments in progress. The one on the right has been learning for ~3 hours, the one in the middle for ~1 hour, and the one on the left just started a few minutes ago. The initial motivation for making the physical Atari was just to commit ourselves to a subset of algorithms that can make progress in this setup. This commitment rules out algorithms that require billions of samples to learn (or worse, require multiple environments running in parallel). Atari games are simple enough that we should be able to show learning on them in a short amount of time with no prior knowledge. Since then, I've realized that this setup is also a good way to compare different paradigms in robotics in a principled way. These paradigms are sim2real, learning from tele-operated data, and learning directly on the robots. So far, I have observed that getting sim2real to work reliably is hard. It requires tweaks that don't scale. Policies that can play perfectly in simulation fall apart because of latencies and the messiness of the real world. These aspects could be modeled to improve the simulation, but not without sinking significant human engineering hours. I have higher hopes for learning from tele-operated data, but that requires a human to learn the task first. These experiments are on my to-do list. I have to learn to play some of the games well through the robot. I’m half-decent at playing Pong and Ms Pacman now. Learning directly on robots is looking like the most promising approach. This approach takes away pesky distribution shifts and makes it possible to have algorithms that continually improve with more data and time without any human intervention. It feels great to let experiments run overnight and wake up to find improved policies. With learning on robots, I should, in principle, be able to go on a long vacation and come back to find better policies for complex tasks beyond Atari games. Whether that is possible with current learning algorithms is a different question.

Experiments in progress. The one on the right has been learning for ~3 hours, the one in the middle for ~1 hour, and the one on the left just started a few minutes ago. The initial motivation for making the physical Atari was just to commit ourselves to a subset of algorithms that can make progress in this setup. This commitment rules out algorithms that require billions of samples to learn (or worse, require multiple environments running in parallel). Atari games are simple enough that we should be able to show learning on them in a short amount of time with no prior knowledge. Since then, I've realized that this setup is also a good way to compare different paradigms in robotics in a principled way. These paradigms are sim2real, learning from tele-operated data, and learning directly on the robots. So far, I have observed that getting sim2real to work reliably is hard. It requires tweaks that don't scale. Policies that can play perfectly in simulation fall apart because of latencies and the messiness of the real world. These aspects could be modeled to improve the simulation, but not without sinking significant human engineering hours. I have higher hopes for learning from tele-operated data, but that requires a human to learn the task first. These experiments are on my to-do list. I have to learn to play some of the games well through the robot. I’m half-decent at playing Pong and Ms Pacman now. Learning directly on robots is looking like the most promising approach. This approach takes away pesky distribution shifts and makes it possible to have algorithms that continually improve with more data and time without any human intervention. It feels great to let experiments run overnight and wake up to find improved policies. With learning on robots, I should, in principle, be able to go on a long vacation and come back to find better policies for complex tasks beyond Atari games. Whether that is possible with current learning algorithms is a different question.

Khurram Javed

52,110 görüntüleme • 7 ay önce

capability != learning new benchtalks with Parth Asawa on continual learning, where we discuss teaching models to learn from experience, measuring learning ability, the bet on parametric models, and more 01:06 What is continual learning? 04:10 Why capability and learning are different 06:13 Why build a benchmark? 08:07 Continual Learning Bench launch and reception 09:13 Anthropic's Fable release and Continual Learning Bench 11:02 How to design tasks for continual learning 18:41 The gain metric 24:01 What good looks like on the leaderboard 29:13 Failure modes: why models can't update their beliefs 31:12 Parametric systems and future architectures 34:30 Open science and AI safety 45:42 Lightning round 49:08 How to contribute to Continual Learning Bench

capability != learning new benchtalks with Parth Asawa on continual learning, where we discuss teaching models to learn from experience, measuring learning ability, the bet on parametric models, and more 01:06 What is continual learning? 04:10 Why capability and learning are different 06:13 Why build a benchmark? 08:07 Continual Learning Bench launch and reception 09:13 Anthropic's Fable release and Continual Learning Bench 11:02 How to design tasks for continual learning 18:41 The gain metric 24:01 What good looks like on the leaderboard 29:13 Failure modes: why models can't update their beliefs 31:12 Parametric systems and future architectures 34:30 Open science and AI safety 45:42 Lightning round 49:08 How to contribute to Continual Learning Bench

vincent sunn chen

39,598 görüntüleme • 15 gün önce

.Richard Sutton, father of reinforcement learning, doesn’t think LLMs are bitter-lesson-pilled. My steel man of Richard’s position: we need some new architecture to enable continual (on-the-job) learning. And if we have continual learning, we don't need a special training phase - the agent just learns on-the-fly - like all humans, and indeed, like all animals. This new paradigm will render our current approach with LLMs obsolete. I did my best to represent the view that LLMs will function as the foundation on which this experiential learning can happen. Some sparks flew. 0:00:00 – Are LLMs a dead-end? 0:13:51 – Do humans do imitation learning? 0:23:57 – The Era of Experience 0:34:25 – Current architectures generalize poorly out of distribution 0:42:17 – Surprises in the AI field 0:47:28 – Will The Bitter Lesson still apply after AGI? 0:54:35 – Succession to AI

.Richard Sutton, father of reinforcement learning, doesn’t think LLMs are bitter-lesson-pilled. My steel man of Richard’s position: we need some new architecture to enable continual (on-the-job) learning. And if we have continual learning, we don't need a special training phase - the agent just learns on-the-fly - like all humans, and indeed, like all animals. This new paradigm will render our current approach with LLMs obsolete. I did my best to represent the view that LLMs will function as the foundation on which this experiential learning can happen. Some sparks flew. 0:00:00 – Are LLMs a dead-end? 0:13:51 – Do humans do imitation learning? 0:23:57 – The Era of Experience 0:34:25 – Current architectures generalize poorly out of distribution 0:42:17 – Surprises in the AI field 0:47:28 – Will The Bitter Lesson still apply after AGI? 0:54:35 – Succession to AI

Dwarkesh Patel

3,080,205 görüntüleme • 9 ay önce

Lots of people think it's obvious that humans learn by imitation, starting with babies imitating their parents. Richard Sutton thinks this is completely wrong. Sutton basically thinks direct imitation plays almost no role in animal learning, and at most a small role in human intelligence. When one animal ends up doing the same thing as another, he thinks this is the result of RL-style trial-and-error, not direct imitation. And if imitation isn't really a kind of learning, then training LLMs to imitate humans isn't really Bitter Lesson–pilled.

Lots of people think it's obvious that humans learn by imitation, starting with babies imitating their parents. Richard Sutton thinks this is completely wrong. Sutton basically thinks direct imitation plays almost no role in animal learning, and at most a small role in human intelligence. When one animal ends up doing the same thing as another, he thinks this is the result of RL-style trial-and-error, not direct imitation. And if imitation isn't really a kind of learning, then training LLMs to imitate humans isn't really Bitter Lesson–pilled.

Dwarkesh Patel

54,245 görüntüleme • 28 gün önce

These are not CGI. Reinforcement learning is so back. When operating on strings, it gives us o3. When operating on physical motors, it gives us a perfect humanoid backflip and a robot creature that out-maneuvers almost every animal on earth. RL is one of the only learning algorithms that can master both the world of bits and the world of atoms. Give me a reward function, and I shall move the world. 2025, Year of RL.

These are not CGI. Reinforcement learning is so back. When operating on strings, it gives us o3. When operating on physical motors, it gives us a perfect humanoid backflip and a robot creature that out-maneuvers almost every animal on earth. RL is one of the only learning algorithms that can master both the world of bits and the world of atoms. Give me a reward function, and I shall move the world. 2025, Year of RL.

Jim Fan

356,747 görüntüleme • 1 yıl önce

Robotics has changed dramatically over the last eight years. Ted Xiao has been involved in the cutting edge of robot learning through this period, spending those eight years at Google Brain/Google Deepmind. And he’s identified three eras of robot learning. These eras are: - The Era of Existence Proofs - trying different methods like QT-Opt, on-robot RL - The Era of Foundation Models - transitioning to data collection and clean objectives (i.e. supervised learning) - The Era of Scaling - orders of magnitude more data and larger models, enabling reasoning, long-horizon actions, and cross-embodiment transfer Watch Episode 78 of RoboPapers, with Michael Cho - Rbt/Acc and Jiafei Duan to learn more!

Robotics has changed dramatically over the last eight years. Ted Xiao has been involved in the cutting edge of robot learning through this period, spending those eight years at Google Brain/Google Deepmind. And he’s identified three eras of robot learning. These eras are: - The Era of Existence Proofs - trying different methods like QT-Opt, on-robot RL - The Era of Foundation Models - transitioning to data collection and clean objectives (i.e. supervised learning) - The Era of Scaling - orders of magnitude more data and larger models, enabling reasoning, long-horizon actions, and cross-embodiment transfer Watch Episode 78 of RoboPapers, with Michael Cho - Rbt/Acc and Jiafei Duan to learn more!

RoboPapers

36,520 görüntüleme • 2 ay önce

WHERE THE ENTERPRISE AI STACK IS BEING BUILT 1. $PLTR building the orchestration layer of the AI economy through the control systems enterprises need before models can take real-world actions backed by a Rule of 40 score of 127 2. $CRWD, $PANW, $ZS & $RBRK building the security layer of the AI economy across endpoint, identity, Zero Trust & cyber recovery as every new agent, machine identity & AI-generated workflow expands the attack surface 3. $NET building the connectivity layer of the AI economy through the internet control plane that agents, APIs & inference traffic must move across in real time, with Cloudflare sitting behind over 20% of the web 4. $SNOW & $MDB building the data layer of the AI economy through the clean, structured, production-grade enterprise data systems that make AI usable inside real applications and workflows

WHERE THE ENTERPRISE AI STACK IS BEING BUILT 1. $PLTR building the orchestration layer of the AI economy through the control systems enterprises need before models can take real-world actions backed by a Rule of 40 score of 127 2. $CRWD, $PANW, $ZS & $RBRK building the security layer of the AI economy across endpoint, identity, Zero Trust & cyber recovery as every new agent, machine identity & AI-generated workflow expands the attack surface 3. $NET building the connectivity layer of the AI economy through the internet control plane that agents, APIs & inference traffic must move across in real time, with Cloudflare sitting behind over 20% of the web 4. $SNOW & $MDB building the data layer of the AI economy through the clean, structured, production-grade enterprise data systems that make AI usable inside real applications and workflows

Shay Boloor

397,599 görüntüleme • 2 ay önce

at Niantic Spatial 🌎, we're building the real-world model for physical AI. we believe the next generation of AI will move beyond the screen, into real-world environments where some of the hardest problems need to be solved. with high-fidelity gaussian splats as the foundation – robots, AI agents, & autonomous systems can see, localize, understand, & operate in the environments that matter most.

at Niantic Spatial 🌎, we're building the real-world model for physical AI. we believe the next generation of AI will move beyond the screen, into real-world environments where some of the hardest problems need to be solved. with high-fidelity gaussian splats as the foundation – robots, AI agents, & autonomous systems can see, localize, understand, & operate in the environments that matter most.

asim ᯅ

42,197 görüntüleme • 2 ay önce

AI agents never stop learning and innovating. With seamless access to interoperable specialized Avalanche L1s, they’ll drive breakthroughs beyond imagination. Breevie shared insights on the new wave of AI-driven innovation. Learn more 👇

AI agents never stop learning and innovating. With seamless access to interoperable specialized Avalanche L1s, they’ll drive breakthroughs beyond imagination. Breevie shared insights on the new wave of AI-driven innovation. Learn more 👇

Avalanche🔺

18,629 görüntüleme • 1 yıl önce

ELON: AI HAS DRAINED HUMAN KNOWLEDGE DRY - NOW DRINKING FROM ITS OWN WELL "You take the entire internet, all books ever written, all the interesting videos, and you distill that down into tokens, essentially bits of information. We've now exhausted all of that; the cumulative sum of human knowledge has been exhausted in AI training. That happened last year. So the only way to supplement that is with synthetic data, where the AI creates. It'll write an essay, or it'll come up with a thesis, and then it will grade itself. It'll go through this process of self-learning with synthetic data." Source: Elon Musk, Mark Penn, January 9th 2025

ELON: AI HAS DRAINED HUMAN KNOWLEDGE DRY - NOW DRINKING FROM ITS OWN WELL "You take the entire internet, all books ever written, all the interesting videos, and you distill that down into tokens, essentially bits of information. We've now exhausted all of that; the cumulative sum of human knowledge has been exhausted in AI training. That happened last year. So the only way to supplement that is with synthetic data, where the AI creates. It'll write an essay, or it'll come up with a thesis, and then it will grade itself. It'll go through this process of self-learning with synthetic data." Source: Elon Musk, Mark Penn, January 9th 2025

Mario Nawfal

1,938,603 görüntüleme • 8 ay önce

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes evencheaper with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,346 görüntüleme • 1 yıl önce

WE'VE BEEN LEARNING WRONG FOR 100 YEARS.. we don't learn from walls of text.. we learn from stories, visuals, interaction Gauth: AI Study Companion just proved it → multi-modal AI that turns the history of the World Cup into a visual world you can actually explore. Try yourself:

WE'VE BEEN LEARNING WRONG FOR 100 YEARS.. we don't learn from walls of text.. we learn from stories, visuals, interaction Gauth: AI Study Companion just proved it → multi-modal AI that turns the history of the World Cup into a visual world you can actually explore. Try yourself:

Robin Delta

50,503 görüntüleme • 1 ay önce

Bernoulli’s got some moves!! Part of the level 3 class is learning to move with your human, and this move is called SWISH!

Bernoulli’s got some moves!! Part of the level 3 class is learning to move with your human, and this move is called SWISH!

Bunsen, Beaker, Bernoulli & Brix

17,057 görüntüleme • 1 ay önce

AI Agents will be the backbone of finance. INFINIT will bring AI Agents beyond DeFi through integrations with fintech and TradFi platforms, reshaping the financial system with agentic coordination. Hear INFINIT's founder Tascha (building @Infinit_Labs) share our vision from @blocmatesdotcom podcast.

AI Agents will be the backbone of finance. INFINIT will bring AI Agents beyond DeFi through integrations with fintech and TradFi platforms, reshaping the financial system with agentic coordination. Hear INFINIT's founder Tascha (building @Infinit_Labs) share our vision from @blocmatesdotcom podcast.

INFINIT

54,628 görüntüleme • 8 ay önce