Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Watch this robot dog learn to walk from scratch in real time! Our new method, APRL, dynamically adjusts exploration constraints to enable fast and performant RL directly in the real world. APRL can also adapt to changes in the terrain. No simulation, no demos. A thread 👇

Sergey Levine

131,940 subscribers

105,579 views • 2 years ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

10 Comments

Sergey Levine2 years ago

The idea in APRL is to adjust action exploration constraints, so that when the policy is bad, the robot performs more conservative exploration, falls less, and explores more effectively. As the policy gets better, the constraints are relaxed and exploration is more aggressive.

Sergey Levine2 years ago

The bounds can be adjusted adaptively using a model misprediction term, which measures how unfamiliar the situation for the robot is. This has the added benefit that when the robot is placed in a new environment, it can automatically modulate exploration to adapt quickly.

Sergey Levine2 years ago

I think it's quite exciting to see real-world RL learn such effective gaits in just 10s of minutes! Our project video provides a more complete explanation of the method:

Sergey Levine2 years ago

To learn more, check out the project website with paper and code here: by @smithlaura1028 & Yunhao Cao

Sergey Levine2 years ago

If you want to learn more about @smithlaura1028's exciting work on real-world RL for robotic locomotion, check out:

malix®️2 years ago

@kevin_zakka Spooky!

James2 years ago

does this make sense on a differential drive wheel robot? eg where it wants to get closer to the goal and just can apply torques to the wheels? I would think with some loss for abrupt changes it would work? BD has a wheeled+legged robot too. would love to see it 'learn' to skate

bSot〰️2 years ago

amazing now do legs

연세쥬피터노트북2 years ago

Wow!!!! I tried this in a year ago and I failed... RL is so difficult without any advisor...

Josh Duyan ☰2 years ago

What effect do you think APRL could have on the field of real-world robotic locomotion?

Related Videos

Qwen-Robot Suite，enable AI from chatbot to physical action in the real world. More demos, please visit our blog:

Qwen-Robot Suite，enable AI from chatbot to physical action in the real world. More demos, please visit our blog:

Qwen

20,129 views • 1 month ago

If you want a robot to do something well, you need to know how to talk to it. If you don't, you can learn, with Semantic Action RL! In our paper, Jagdeep Bhatia, Andrew Wagenmaker, Will Chen show how RL over VLA prompts enables new tasks and learns blazing fast in the real world!

If you want a robot to do something well, you need to know how to talk to it. If you don't, you can learn, with Semantic Action RL! In our paper, Jagdeep Bhatia, Andrew Wagenmaker, Will Chen show how RL over VLA prompts enables new tasks and learns blazing fast in the real world!

Sergey Levine

30,319 views • 1 month ago

The best way to get robust, high-quality robot performance is through reinforcement learning; but RL in either the real world or a traditional simulation has lots of limitations. Instead, Jiazhi Yang in RISE does RL in a compositional world model. Learn more ->

The best way to get robust, high-quality robot performance is through reinforcement learning; but RL in either the real world or a traditional simulation has lots of limitations. Instead, Jiazhi Yang in RISE does RL in a compositional world model. Learn more ->

Chris Paxton

34,046 views • 1 month ago

How to harness foundation models for *generalization in the wild* in robot manipulation? Introducing VoxPoser: use LLM+VLM to label affordances and constraints directly in 3D perceptual space for zero-shot robot manipulation in the real world! 🌐 🧵👇

How to harness foundation models for generalization in the wild in robot manipulation? Introducing VoxPoser: use LLM+VLM to label affordances and constraints directly in 3D perceptual space for zero-shot robot manipulation in the real world! 🌐 🧵👇

Wenlong Huang

293,910 views • 3 years ago

Punchline: distill world models from simulation to enable fast, stable real-world robot adaptation. Simulation is nearly always wrong. But in Simulation Distillation, we ask a simple question: How do we perform simulation pretraining such that real-world adaptation becomes trivially easy? Let's take a closer look (1/n)

Punchline: distill world models from simulation to enable fast, stable real-world robot adaptation. Simulation is nearly always wrong. But in Simulation Distillation, we ask a simple question: How do we perform simulation pretraining such that real-world adaptation becomes trivially easy? Let's take a closer look (1/n)

Abhishek Gupta

32,779 views • 2 months ago

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

Ilir Aliu - eu/acc

10,509 views • 1 year ago

Our model can now learn from its own experience with RL! Our new π*0.6 model can more than double throughput over a base model trained without RL, and can perform real-world tasks: making espresso drinks, folding diverse laundry, and assembling boxes. More in the thread below.

Our model can now learn from its own experience with RL! Our new π*0.6 model can more than double throughput over a base model trained without RL, and can perform real-world tasks: making espresso drinks, folding diverse laundry, and assembling boxes. More in the thread below.

Physical Intelligence

706,891 views • 8 months ago

Demonstrating our latest localization system in action. At bootup, the robot has no prior position estimate, but within seconds it autonomously localizes itself using a fusion of three algorithms. Unlike our previous version, this updated system incorporates vision, allowing the robot to adapt to real-world changes (e.g., moved furniture) by recognizing previously seen environments. In this demo, we repeatedly reset navigation and reposition the robot to random locations, showing robust, repeatable localization. The robot then executes a full patrol, following a planned path (visualized in RViz) with real-time path tracking. Because our software is hardware-agnostic, it brings the same reliable performance to any robot it runs on.

Demonstrating our latest localization system in action. At bootup, the robot has no prior position estimate, but within seconds it autonomously localizes itself using a fusion of three algorithms. Unlike our previous version, this updated system incorporates vision, allowing the robot to adapt to real-world changes (e.g., moved furniture) by recognizing previously seen environments. In this demo, we repeatedly reset navigation and reposition the robot to random locations, showing robust, repeatable localization. The robot then executes a full patrol, following a planned path (visualized in RViz) with real-time path tracking. Because our software is hardware-agnostic, it brings the same reliable performance to any robot it runs on.

OpenMind

23,954 views • 3 months ago

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Abhishek Gupta

10,803 views • 1 year ago

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

SciTech Era

1,584,468 views • 2 months ago

RL in the real world presents some big challenges, but also some really big opportunities. In our new work, HIL-SERL, Charles Xu, Jeffrey Wu, Jianlan Luo show that real-world RL can learn a huge range of precise and robust tasks, and perform them much faster than imitation.

RL in the real world presents some big challenges, but also some really big opportunities. In our new work, HIL-SERL, Charles Xu, Jeffrey Wu, Jianlan Luo show that real-world RL can learn a huge range of precise and robust tasks, and perform them much faster than imitation.

Sergey Levine

36,489 views • 1 year ago

Meet Any2Track, a new two-stage RL framework designed to address a core challenge for humanoid robots: reliably tracking diverse, dynamic motions in the real world. The framework consists of two key components: AnyTracker, a general motion tracker, and AnyAdapter, a module that allows the robot to adapt to real-world disturbances like uneven terrain or external forces. This approach gives robots the stability needed for practical use. The system has been successfully deployed on a Unitree G1 humanoid robot with zero-shot sim2real transfer, performing exceptionally well in various motion tracking tasks under multiple real-world disturbances. Project: Codebase:

Meet Any2Track, a new two-stage RL framework designed to address a core challenge for humanoid robots: reliably tracking diverse, dynamic motions in the real world. The framework consists of two key components: AnyTracker, a general motion tracker, and AnyAdapter, a module that allows the robot to adapt to real-world disturbances like uneven terrain or external forces. This approach gives robots the stability needed for practical use. The system has been successfully deployed on a Unitree G1 humanoid robot with zero-shot sim2real transfer, performing exceptionally well in various motion tracking tasks under multiple real-world disturbances. Project: Codebase:

RoboHub🤖

61,378 views • 10 months ago

Testing robot policies on hardware is slow, expensive and hard to scale. World models offer a promising path to accelerating robot policy development. We're sharing new research from the Runway Robotics team, in which we simulated 8 robot policies inside our General World Model and found 0.95 correlation with real-world results. Those early results point to world model simulation as a practical substitute for hardware evaluation, comparing favorably to existing real-to-sim approaches. Learn more at the link below.

Testing robot policies on hardware is slow, expensive and hard to scale. World models offer a promising path to accelerating robot policy development. We're sharing new research from the Runway Robotics team, in which we simulated 8 robot policies inside our General World Model and found 0.95 correlation with real-world results. Those early results point to world model simulation as a practical substitute for hardware evaluation, comparing favorably to existing real-to-sim approaches. Learn more at the link below.

Runway

13,464 views • 5 months ago

Excited to share the project that has surprised me the most in the last year! Large-scale RL in simulation, no demos and no reward engineering can solve dynamic, dexterous and contact rich tasks. The learned behaviors are reactive, forceful and use the environment for recovery in ways that are extremely challenging to bake in or teleoperate! You can play with the policies yourself to see: And, the learned behavior transfers to real world robots from RGB camera inputs! So what’s the trick - using simulator resets carefully! Let’s unpack (1/10)

Excited to share the project that has surprised me the most in the last year! Large-scale RL in simulation, no demos and no reward engineering can solve dynamic, dexterous and contact rich tasks. The learned behaviors are reactive, forceful and use the environment for recovery in ways that are extremely challenging to bake in or teleoperate! You can play with the policies yourself to see: And, the learned behavior transfers to real world robots from RGB camera inputs! So what’s the trick - using simulator resets carefully! Let’s unpack (1/10)

Abhishek Gupta

82,861 views • 4 months ago

🇨🇳 China moves from robot demos to real deployment. The plan is to deploy 10,000 humanoid robots in more than 100 real-world applications by year-end.

🇨🇳 China moves from robot demos to real deployment. The plan is to deploy 10,000 humanoid robots in more than 100 real-world applications by year-end.

Jackson Hinkle 🇺🇸

50,049 views • 1 month ago

If you want to see what real industrialization looks like (not demos or gimmicks but a real robot fleet on its way out the door to do real work) watch this clip from one of our deployment warehouses tonight. Also if you know of anyone who has extra space in Chicago fast plz let us know we’re bursting st the seams.

If you want to see what real industrialization looks like (not demos or gimmicks but a real robot fleet on its way out the door to do real work) watch this clip from one of our deployment warehouses tonight. Also if you know of anyone who has extra space in Chicago fast plz let us know we’re bursting st the seams.

Saman Farid

11,790 views • 6 months ago

So we did a bunch of projects with real world reinforcement learning - but it was often too inefficient to be practical to train tabula rasa. This suggests we need better priors, but acquiring these from on-robot data can often be expensive as well. In our recent work, we show that despite being fundamentally inaccurate, simulation can guide provide a cheap way to guide real-world RL finetuning to be super efficient! We propose Simulation-Guided Fine-Tuning (SGFT) - a simple paradigm for sim2real finetuning that uses simulation to provide reward shaping that accelerates real world RL finetuning *beyond* just providing an initialization. TLDR: Use value functions from sim to shape rewards for real-world RL, see large sample efficiency improvements 🧵(1/6)

So we did a bunch of projects with real world reinforcement learning - but it was often too inefficient to be practical to train tabula rasa. This suggests we need better priors, but acquiring these from on-robot data can often be expensive as well. In our recent work, we show that despite being fundamentally inaccurate, simulation can guide provide a cheap way to guide real-world RL finetuning to be super efficient! We propose Simulation-Guided Fine-Tuning (SGFT) - a simple paradigm for sim2real finetuning that uses simulation to provide reward shaping that accelerates real world RL finetuning beyond just providing an initialization. TLDR: Use value functions from sim to shape rewards for real-world RL, see large sample efficiency improvements 🧵(1/6)

Abhishek Gupta

13,637 views • 1 year ago

THE FUTURE IS HERE! For the first time ever, you can now create your own 100% AI avatar from scratch. Simply describe your ideal representative, from their appearance to their surroundings, and our technology will bring your vision to life! Available now at: (examples in thread) [No real people were filmed in the making of this video.]

Creatify AI

72,415 views • 1 year ago

🦋 Last week, our team took delivery of a new quadruped robotic dog from Unitree — the Unitree Go 2 AIR. With Letta and the Animus framework, developers can build embodied agents that think, learn, and act in the real world. Watch how we plan to bring Animus to life on this new hardware. Build 🤖 on Animus

🦋 Last week, our team took delivery of a new quadruped robotic dog from Unitree — the Unitree Go 2 AIR. With Letta and the Animus framework, developers can build embodied agents that think, learn, and act in the real world. Watch how we plan to bring Animus to life on this new hardware. Build 🤖 on Animus

Animus

33,084 views • 9 months ago

NVIDIA AI Aerial uses neural networks to learn from data, adapt in real time, and scale with complexity. This boosts spectral efficiency, lowers costs, and improves connectivity for telecom. To learn more, watch the full video ➡️

NVIDIA AI Aerial uses neural networks to learn from data, adapt in real time, and scale with complexity. This boosts spectral efficiency, lowers costs, and improves connectivity for telecom. To learn more, watch the full video ➡️

NVIDIA AI

11,236 views • 1 year ago