Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Can robots practice tasks on their own? We introduce a method for improving autonomy in robotic reinforcement learning. A thread 🧵

Chelsea Finn

103,039 subscribers

63,823 views • 3 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

4 Comments

Chelsea Finn3 years ago

In principle, reinforcement learning can allow robots to learn on their own. In practice, people need to spend significant effort to define rewards and "reset" the world between trials. Can a robot improve by itself?

Chelsea Finn3 years ago

Our approach, MEDAL++, has a simple procedure: 1. Human provides a few demos 2. Simultaneously: - learn a task reward function - learn 2 policies to do and undo the task

Chelsea Finn3 years ago

MEDAL++ allows robots to learn challenging tasks from camera image inputs, with a high degree of autonomy. Some important bits to get it to work: - heavy regularization on learned reward - regularize policy towards demos - wrist-mounted camera - RED-Q (

Chelsea Finn3 years ago

A challenging but rewarding project led by @archit_sharma97, @AhmedSQRD, Rehaan Ahmad Website: Paper: Code:

Related Videos

We introduce a system for fine-grained robotic manipulation! 🤖 What’s new? * We can control cheap robots to do surprisingly dexterous tasks * New technique that allows robots to learn fine motor skills A short thread 🧵

We introduce a system for fine-grained robotic manipulation! 🤖 What’s new? * We can control cheap robots to do surprisingly dexterous tasks * New technique that allows robots to learn fine motor skills A short thread 🧵

Chelsea Finn

264,494 views • 3 years ago

What if robots could improve themselves by learning from their own failures in the real-world? Introducing 𝗣𝗟𝗗 (𝗣𝗿𝗼𝗯𝗲, 𝗟𝗲𝗮𝗿𝗻, 𝗗𝗶𝘀𝘁𝗶𝗹𝗹) — a recipe that enables Vision-Language-Action (VLA) models to self-improve for high-precision manipulation tasks. PLD couples real-world residual reinforcement learning with standard supervised fine-tuning — letting robots discover, recover, and distill their own data flywheel. Quick 🧵

What if robots could improve themselves by learning from their own failures in the real-world? Introducing 𝗣𝗟𝗗 (𝗣𝗿𝗼𝗯𝗲, 𝗟𝗲𝗮𝗿𝗻, 𝗗𝗶𝘀𝘁𝗶𝗹𝗹) — a recipe that enables Vision-Language-Action (VLA) models to self-improve for high-precision manipulation tasks. PLD couples real-world residual reinforcement learning with standard supervised fine-tuning — letting robots discover, recover, and distill their own data flywheel. Quick 🧵

Wenli Xiao

186,079 views • 11 months ago

Today's robots are too unreliable and slow for real industrial work. Reinforcement learning changes that. Meet KinetIQ Ascend: our robots practice real production tasks and improve predictably towards industrial reliability standards. How it works 🧵

Today's robots are too unreliable and slow for real industrial work. Reinforcement learning changes that. Meet KinetIQ Ascend: our robots practice real production tasks and improve predictably towards industrial reliability standards. How it works 🧵

hr0nix

11,052 views • 3 months ago

NeurIPS 2025 Paper: LLMs are Reinforcement Learners 🤯! Surprisingly, we show that LLMs can solve RL tasks without any external component! We introduce Prompted Policy Search (ProPS), an RL method based only LLMs and in-context learning. [Paper]

NeurIPS 2025 Paper: LLMs are Reinforcement Learners 🤯! Surprisingly, we show that LLMs can solve RL tasks without any external component! We introduce Prompted Policy Search (ProPS), an RL method based only LLMs and in-context learning. [Paper]

Heni Ben Amor

51,248 views • 10 months ago

We now have full end-to-end manipulation autonomy for more complicated tasks. Here's a demonstration of Apollo leveraging AI to make a fresh cup of juice in an end-to-end manner for #GTC2024. A thread 🧵

We now have full end-to-end manipulation autonomy for more complicated tasks. Here's a demonstration of Apollo leveraging AI to make a fresh cup of juice in an end-to-end manner for #GTC2024. A thread 🧵

Nathan Boyd 🐙

18,054 views • 2 years ago

NEW: Disney reveals “ReActor,” a new method for Walt Disney Imagineering’s robotic character pipeline. • Combines reinforcement learning with physics-based simulations to transfer human motion to characters • Could be the next milestone for more lifelike robotic characters

NEW: Disney reveals “ReActor,” a new method for Walt Disney Imagineering’s robotic character pipeline. • Combines reinforcement learning with physics-based simulations to transfer human motion to characters • Could be the next milestone for more lifelike robotic characters

Drew Smith

66,859 views • 5 months ago

How can we get VLMs to move their eyes—and reason step-by-step in visually grounded ways? 👀 We introduce ViGoRL, a RL method that anchors reasoning to image regions. 🎯 It outperforms vanilla GRPO and SFT across grounding, spatial tasks, and visual search (86.4% on V*). 👇🧵

How can we get VLMs to move their eyes—and reason step-by-step in visually grounded ways? 👀 We introduce ViGoRL, a RL method that anchors reasoning to image regions. 🎯 It outperforms vanilla GRPO and SFT across grounding, spatial tasks, and visual search (86.4% on V*). 👇🧵

Gabriel Sarch

76,824 views • 1 year ago

notes on my reinforcement learning for rocket league side quest so far: i’m in awe that — we live in a time where i can train a reinforcement learning model that is learning to play rocket league and at the same time, test community made rocket league models. where, both processes can run in parallel & locally on my hardware. in the game window — i’m playing against a reinforcement learning ppo model that’s running locally. and in the vscode window — i’m training my own reinforcement learning model (just a dummy example script for now). also, the rocket league botting ecosystem has incredible tooling where you can test out different reinforcement learning ideas against other algorithms + it has great documentation for linux and windows. it’s honestly a beautiful hidden gem of a rabbit hole :)

notes on my reinforcement learning for rocket league side quest so far: i’m in awe that — we live in a time where i can train a reinforcement learning model that is learning to play rocket league and at the same time, test community made rocket league models. where, both processes can run in parallel & locally on my hardware. in the game window — i’m playing against a reinforcement learning ppo model that’s running locally. and in the vscode window — i’m training my own reinforcement learning model (just a dummy example script for now). also, the rocket league botting ecosystem has incredible tooling where you can test out different reinforcement learning ideas against other algorithms + it has great documentation for linux and windows. it’s honestly a beautiful hidden gem of a rabbit hole :)

naklecha

46,795 views • 1 year ago

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

18,813 views • 9 months ago

Diffusion models make great images. But can they drive robots? Usually that gets complicated really fast. We figured out how to get a Stable Diffusion model (based on Instruct pix2pix) to drive robotic instruction following. Simple recipe, works on a wide range of tasks. Thread👇

Diffusion models make great images. But can they drive robots? Usually that gets complicated really fast. We figured out how to get a Stable Diffusion model (based on Instruct pix2pix) to drive robotic instruction following. Simple recipe, works on a wide range of tasks. Thread👇

Sergey Levine

126,572 views • 3 years ago

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning investigated the application of Deep Reinforcement Learning (Deep RL) for low-cost, miniature humanoid hardware in a dynamic environment, showing the method can synthesize sophisticated and safe movement skills making up complex behavioral strategies in a simplified one-versus-one (1v1) soccer game abs: project page:

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning investigated the application of Deep Reinforcement Learning (Deep RL) for low-cost, miniature humanoid hardware in a dynamic environment, showing the method can synthesize sophisticated and safe movement skills making up complex behavioral strategies in a simplified one-versus-one (1v1) soccer game abs: project page:

AK

293,474 views • 3 years ago

I trained a humanoid robot to play famous anime openings on the piano with reinforcement learning (Attack on Titan, Evangelion, SAO, and more) how I did it + surprising discoveries in thread🧵

I trained a humanoid robot to play famous anime openings on the piano with reinforcement learning (Attack on Titan, Evangelion, SAO, and more) how I did it + surprising discoveries in thread🧵

anandmaj

24,740 views • 1 year ago

In the future we will have AI powered Robotic Kids They will never grow old, always keep you company… because they are robots. Disney will sell them to you 😱 🧵 A Thread

In the future we will have AI powered Robotic Kids They will never grow old, always keep you company… because they are robots. Disney will sell them to you 😱 🧵 A Thread

Linus ✦ Ekenstam

301,099 views • 3 years ago

While Unitree sells a lot of dumb humanoid robots to developers and researchers, they're also working on their own AI models. Robots making robots: Unitree’s UnifoLM-X1-0 AI model in action, performing assembly tasks in Unitree's own factory.

While Unitree sells a lot of dumb humanoid robots to developers and researchers, they're also working on their own AI models. Robots making robots: Unitree’s UnifoLM-X1-0 AI model in action, performing assembly tasks in Unitree's own factory.

The Humanoid Hub

26,867 views • 8 months ago

Analyzing mavies is a great way of improving your own compositions. See thread for more layout and storyboard tips! 1/8 🧵

Analyzing mavies is a great way of improving your own compositions. See thread for more layout and storyboard tips! 1/8 🧵

Mitch Leeuwe

1,938,450 views • 3 years ago

Robots help us do tasks in space that are too dangerous or challenging for humans. We’re working on a new generation of robotic spacecraft that can work together without being controlled by astronauts or teams on Earth:

Robots help us do tasks in space that are too dangerous or challenging for humans. We’re working on a new generation of robotic spacecraft that can work together without being controlled by astronauts or teams on Earth:

NASA

901,241 views • 3 years ago

🧵Robots have learned from demos. They’ve mimicked humans. But they’ve rarely improved themselves. π*0.6 may change that, it learns like we do: through instruction, coaching, and practice. A real step forward for general-purpose robots. [Save this thread for later 📌]

🧵Robots have learned from demos. They’ve mimicked humans. But they’ve rarely improved themselves. π*0.6 may change that, it learns like we do: through instruction, coaching, and practice. A real step forward for general-purpose robots. [Save this thread for later 📌]

Lukas Ziegler

10,667 views • 10 months ago

Can VLMs enable robots to autonomously improve? In our new work we ran a fleet of robot arms to collect autonomous data with VLM-proposed tasks and showed that robots can keep getting better as they are deployed, without supervision: 🧵👇

Can VLMs enable robots to autonomously improve? In our new work we ran a fleet of robot arms to collect autonomous data with VLM-proposed tasks and showed that robots can keep getting better as they are deployed, without supervision: 🧵👇

Sergey Levine

19,449 views • 2 years ago

Excited to share my final PhD project😀 We show how simple, yet elegant changes enable diffusion transformers to learn SOTA robotic policies on real robots. Our method improves performance by 20% across a wide range of highly dexterous tasks - like cutting sushi! 1/n

Excited to share my final PhD project😀 We show how simple, yet elegant changes enable diffusion transformers to learn SOTA robotic policies on real robots. Our method improves performance by 20% across a wide range of highly dexterous tasks - like cutting sushi! 1/n

Sudeep Dasari

20,536 views • 2 years ago

Does your sim2real robot falter at critical moments 🤯? Want to help but unsure how, all you can do is reward tuning in sim 😮‍💨? Introduce 𝐓𝐑𝐀𝐍𝐒𝐈𝐂 for manipulation sim2real. Robots learned in sim can accomplish complex tasks in real, such as furniture assembly. 🤿🧵

Does your sim2real robot falter at critical moments 🤯? Want to help but unsure how, all you can do is reward tuning in sim 😮‍💨? Introduce 𝐓𝐑𝐀𝐍𝐒𝐈𝐂 for manipulation sim2real. Robots learned in sim can accomplish complex tasks in real, such as furniture assembly. 🤿🧵

Yunfan Jiang

43,229 views • 2 years ago