Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Mondo's Beni ( may look cute, but there is a serious learning stack inside. It can perform flips, navigate autonomously, and track moving subjects. The secret is reinforcement learning: Beni practices at massive scale in simulation, optimizing its controller through millions of virtual trials before entering the real world.... show more

Shuo Yang

5,468 subscribers

31,539 views • 3 months ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

24 Comments

Shuo Yang3 months ago

1/ @nvidia IsaacLab and @GoogleDeepMind MuJoCo are core parts of our training and validation pipeline. We train controllers across large numbers of simulated environments, then validate them in a separate physics engine.

Shuo Yang3 months ago

2/ Modern legged-robot reinforcement learning builds on years of excellent research. The work of @rdn_nikita , David Hoeller, and their collaborators has been particularly influential. For Beni’s flipping motions, we also drew inspiration from @xbpeng4’s classic DeepMimic paper. Beni has been reading the literature—and practicing its backflips.

Shuo Yang3 months ago

3/ Some behaviors need an additional layer of sim-to-real polish. For these, we use ideas from Regularized Online Adaptation, or ROA, developed by my good friends @zipengfu and @xuxin_cheng (and @deepakpathak) :

Shuo Yang3 months ago

4/ Along side all the neural networks, Beni also uses some good old-fashioned SLAM—in several creative ways. One particularly valuable tool is GTSAM, created by @fdellaert and contributors.

Shuo Yang3 months ago

5/ Beni’s target-tracking system also contains several algorithmic innovations developed by our own team. These improvements allow Beni to follow a subject smoothly while coordinating perception, locomotion, and camera motion. We may publish more technical details in the future.

Shuo Yang3 months ago

Sometimes the best robotics system is not “AI versus classical robotics.” It is AI standing on top of decades of great robotics research. Small robot. Serious algorithms. 🤖

Carlos DP 🤖🇺🇸3 months ago

is there a developer mode on Beni? like, could he be used for people learning to do RL on real robots in a fun way?

Deepak Pathak3 months ago

Great work @Shuo — love to see our ROA algorithm in action. This robot is so cool!

JVM3 months ago

and you want $549 for this?!

cr_ss_2 months ago

Thats a really cool toy!

HGONG | Founder & Researcher3 months ago

咦？你是RM的杨硕吗？

Ankur Deka3 months ago

Great to see Beni’s acrobatics @ShuoYangAIR!

Keith Botha3 months ago

We love to see it ! Really good partnership opportunity with @UFBots

Jrmginer3 months ago

that’s cool！

Swiftlined3 months ago

Solid stuff

tobitege3 months ago

Now I want Beni to kick back 🤣 Love that lil guy, even though I don't have a use for him!

Will Kriski2 months ago

can we train it ourselves similar to microduck and use a simulator? @pollenrobotics

theobot3 months ago

this is awesome

Sadikk2 months ago

This is great for cat parents.. Best toy for cats to ay with.. Consider cheaper version.

Tanay Jaipuria3 months ago

have you experimented with world models for simulation?

Black Eyed Hat Man3 months ago

😈 cute! But was the kick necessary?! 😰

T.3 months ago

a cutie, how hes following the skater 🥰

AI Apps API3 months ago

The simulation scale is the impressive part, and it is also where the hard problem hides. Training at that volume gets you a policy that is optimal for your simulator, including its contact model, its friction constants and its integration step. The gap to hardware is not noise, it is systematic, and domain randomization tends to paper over it rather than measure it. The number I always want to see alongside these is how much the policy degrades on the real robot before any fine tuning. That one figure tells you how good the physics model actually was.

Hoon_Sky3 months ago

Probably cannot last long on batteries

Related Videos

The RL era of robotics is here, and simulation is how to scale it. An early look at our research on a achieving a better scaling paradigm for robot learning with simulation 👇

The RL era of robotics is here, and simulation is how to scale it. An early look at our research on a achieving a better scaling paradigm for robot learning with simulation 👇

Moonlake

474,388 views • 5 days ago

This is a demonstration video of in-hand object rotation with new Torobo Hand. By leveraging large-scale parallel reinforcement learning in NVIDIA Isaac Sim, the learned policy can perform object manipulation in MuJoCo and on the real robot without any additional training.

This is a demonstration video of in-hand object rotation with new Torobo Hand. By leveraging large-scale parallel reinforcement learning in NVIDIA Isaac Sim, the learned policy can perform object manipulation in MuJoCo and on the real robot without any additional training.

東京ロボティクス株式会社

10,951 views • 1 year ago

The best way to get robust, high-quality robot performance is through reinforcement learning; but RL in either the real world or a traditional simulation has lots of limitations. Instead, Jiazhi Yang in RISE does RL in a compositional world model. Learn more ->

The best way to get robust, high-quality robot performance is through reinforcement learning; but RL in either the real world or a traditional simulation has lots of limitations. Instead, Jiazhi Yang in RISE does RL in a compositional world model. Learn more ->

Chris Paxton

34,102 views • 5 months ago

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 views • 1 year ago

Today we present Morpheus, a persistent enterprise simulation platform designed to make Continual Learning a reality. Morpheus is the world’s first real world Reinforcement Learning environment. Every Reinforcement Learning environment operates in the game world. Benchmarks like Atari, OpenAI Gym, MuJoCo, and Procgen are all small, game-like worlds that reset every few minutes. But the real world never resets. A business keeps running and evolving everyday. We tested how frontier LLMs would perform in realistic and dynamic business environments 🧬on Morpheus. The main conclusion was that LLMs are not continual learners. 🧵Here’s how we did it and what we learned:

Today we present Morpheus, a persistent enterprise simulation platform designed to make Continual Learning a reality. Morpheus is the world’s first real world Reinforcement Learning environment. Every Reinforcement Learning environment operates in the game world. Benchmarks like Atari, OpenAI Gym, MuJoCo, and Procgen are all small, game-like worlds that reset every few minutes. But the real world never resets. A business keeps running and evolving everyday. We tested how frontier LLMs would perform in realistic and dynamic business environments 🧬on Morpheus. The main conclusion was that LLMs are not continual learners. 🧵Here’s how we did it and what we learned:

Skyfall AI

270,028 views • 4 months ago

Just months after its debut, Atlas is proving why it is the world’s most capable and dynamic humanoid robot, ready for real work. Lifting a mini-fridge is a feat of strength, but the true breakthrough is in the underlying reinforcement learning and controls systems.

Just months after its debut, Atlas is proving why it is the world’s most capable and dynamic humanoid robot, ready for real work. Lifting a mini-fridge is a feat of strength, but the true breakthrough is in the underlying reinforcement learning and controls systems.

Boston Dynamics

41,726 views • 5 months ago

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,867 views • 5 months ago

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

18,813 views • 10 months ago

Bittensor is the main thing I’m watching at Proof of Talk. The subnet conversation is moving from CT speculation into real institutional interest now. Decentralized intelligence, open markets for machine learning, subnet economies, this is the part of crypto that actually feels like it can matter at scale. Most people still underestimate how big that shift is. Feels like $TAO is entering its serious capital era.

Bittensor is the main thing I’m watching at Proof of Talk. The subnet conversation is moving from CT speculation into real institutional interest now. Decentralized intelligence, open markets for machine learning, subnet economies, this is the part of crypto that actually feels like it can matter at scale. Most people still underestimate how big that shift is. Feels like $TAO is entering its serious capital era.

nordin.eth

11,217 views • 5 months ago

See Spot perform dynamic whole-body manipulation. Using a combination of reinforcement learning (RL) and sampling-based control, the robot is able to autonomously drag, roll, and stack tires weighing 15 kg (33 lb), well above its maximum arm lift capacity. Learn more about coordinating locomotion and manipulation processes:

See Spot perform dynamic whole-body manipulation. Using a combination of reinforcement learning (RL) and sampling-based control, the robot is able to autonomously drag, roll, and stack tires weighing 15 kg (33 lb), well above its maximum arm lift capacity. Learn more about coordinating locomotion and manipulation processes:

RAI Institute

87,753 views • 1 year ago

Reinforcement learning is used to speed the production of behavior for the Boston Dynamics Atlas humanoid robot. At the heart of the learning process is a physics-based simulator that generates training data for a variety of maneuvers.

Reinforcement learning is used to speed the production of behavior for the Boston Dynamics Atlas humanoid robot. At the heart of the learning process is a physics-based simulator that generates training data for a variety of maneuvers.

RAI Institute

76,796 views • 1 year ago

notes on my reinforcement learning for rocket league side quest so far: i’m in awe that — we live in a time where i can train a reinforcement learning model that is learning to play rocket league and at the same time, test community made rocket league models. where, both processes can run in parallel & locally on my hardware. in the game window — i’m playing against a reinforcement learning ppo model that’s running locally. and in the vscode window — i’m training my own reinforcement learning model (just a dummy example script for now). also, the rocket league botting ecosystem has incredible tooling where you can test out different reinforcement learning ideas against other algorithms + it has great documentation for linux and windows. it’s honestly a beautiful hidden gem of a rabbit hole :)

notes on my reinforcement learning for rocket league side quest so far: i’m in awe that — we live in a time where i can train a reinforcement learning model that is learning to play rocket league and at the same time, test community made rocket league models. where, both processes can run in parallel & locally on my hardware. in the game window — i’m playing against a reinforcement learning ppo model that’s running locally. and in the vscode window — i’m training my own reinforcement learning model (just a dummy example script for now). also, the rocket league botting ecosystem has incredible tooling where you can test out different reinforcement learning ideas against other algorithms + it has great documentation for linux and windows. it’s honestly a beautiful hidden gem of a rabbit hole :)

naklecha

46,795 views • 1 year ago

Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma. To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference.

Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma. To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference.

Carlo Sferrazza

220,153 views • 11 months ago

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

SciTech Era

1,585,680 views • 5 months ago

These are not CGI. Reinforcement learning is so back. When operating on strings, it gives us o3. When operating on physical motors, it gives us a perfect humanoid backflip and a robot creature that out-maneuvers almost every animal on earth. RL is one of the only learning algorithms that can master both the world of bits and the world of atoms. Give me a reward function, and I shall move the world. 2025, Year of RL.

These are not CGI. Reinforcement learning is so back. When operating on strings, it gives us o3. When operating on physical motors, it gives us a perfect humanoid backflip and a robot creature that out-maneuvers almost every animal on earth. RL is one of the only learning algorithms that can master both the world of bits and the world of atoms. Give me a reward function, and I shall move the world. 2025, Year of RL.

Jim Fan

357,071 views • 1 year ago

Mark Zuckerberg and Priscilla Chan say virtual cells could change how science takes risk, letting researchers test bold ideas in simulation before entering the wet lab. “The promise of being able to do virtual biology… is you can actually take on riskier ideas.” “If you had a virtual cell model… you could actually then start testing it and tinkering on the computational side… and actually see if there is promise doing the experiments in silico before you make the time and money investment in the wet lab.”

Mark Zuckerberg and Priscilla Chan say virtual cells could change how science takes risk, letting researchers test bold ideas in simulation before entering the wet lab. “The promise of being able to do virtual biology… is you can actually take on riskier ideas.” “If you had a virtual cell model… you could actually then start testing it and tinkering on the computational side… and actually see if there is promise doing the experiments in silico before you make the time and money investment in the wet lab.”

a16z

44,009 views • 1 year ago

Former Meta Chief AI Scientist Yann LeCun on the three paradigms of machine learning — and why the third is what made ChatGPT possible: Here's each one, and where it breaks. First, supervised learning. You tell the machine the answer. "You show it a picture, let's say of a table, and you tell it this is a table. So it's supervised because you tell it what the correct answer is." Get it wrong, and the machine rewrites itself: "The system computes its output, and if it says something else than table, then it's going to adjust its parameters, its internal structure, so that the output it produces gets closer to the output you want." Repeat at scale and something more than memorisation appears: "Eventually the system will find a way to recognize every image you trained it on, but also images it's never seen that are similar to the one you train it on. This is called a generalization ability." The limit: a human has to supply every single answer. That doesn't scale to the size of the internet. Second, reinforcement learning. You don't give the answer, only a verdict. "You don't tell the system what the correct answer is. You only tell it whether the answer it produced was good or bad." Learning to ride a bike, essentially: "You try to ride a bike and you don't know how to ride the bike and after a while you fall. So you know you did something bad and so you change your strategy a little bit. And eventually you learn how to ride a bike." For years the field assumed this was the closest thing to how animals actually learn. Yann LeCun's verdict: "Now it turns out reinforcement learning is extremely inefficient." It dominates wherever failure is free: "It works really well if you want to train a system to play chess or play go or poker, because you can have the system play millions and millions of games against itself and basically fine-tune itself. But it doesn't really work in the real world." The limit, in one image: "If you want to train a car to drive itself, you're not going to do it with reinforcement learning. It's going to crash thousands of times." On robotics he's careful rather than dismissive: "Reinforcement learning can be part of the solution, but it's not the complete answer. It's not sufficient." Third, self-supervised learning. You tell the machine nothing at all. "And this is what has enabled the recent progress in natural language understanding and chatbots." The strange part is that you stop asking for a task: "You don't train the system to accomplish any particular task. You just train it to basically capture the structure..." The method is deliberate sabotage: "You take a piece of text, you corrupt it in some way, by for example removing some words, and then you train a big neural net to predict the words that are missing." And one narrow version of that trick runs every chatbot on Earth: "A special case of this is that you take a piece of text and the last word in that text is not visible, and so you train the system to predict the last word in that text — and this is the way large language models are trained on." So why did the third one win? Supervised learning needs a human. Reinforcement learning needs a crash. Self-supervised learning needs neither — because the missing word and the correct answer are the same thing. The data grades itself.

Former Meta Chief AI Scientist Yann LeCun on the three paradigms of machine learning — and why the third is what made ChatGPT possible: Here's each one, and where it breaks. First, supervised learning. You tell the machine the answer. "You show it a picture, let's say of a table, and you tell it this is a table. So it's supervised because you tell it what the correct answer is." Get it wrong, and the machine rewrites itself: "The system computes its output, and if it says something else than table, then it's going to adjust its parameters, its internal structure, so that the output it produces gets closer to the output you want." Repeat at scale and something more than memorisation appears: "Eventually the system will find a way to recognize every image you trained it on, but also images it's never seen that are similar to the one you train it on. This is called a generalization ability." The limit: a human has to supply every single answer. That doesn't scale to the size of the internet. Second, reinforcement learning. You don't give the answer, only a verdict. "You don't tell the system what the correct answer is. You only tell it whether the answer it produced was good or bad." Learning to ride a bike, essentially: "You try to ride a bike and you don't know how to ride the bike and after a while you fall. So you know you did something bad and so you change your strategy a little bit. And eventually you learn how to ride a bike." For years the field assumed this was the closest thing to how animals actually learn. Yann LeCun's verdict: "Now it turns out reinforcement learning is extremely inefficient." It dominates wherever failure is free: "It works really well if you want to train a system to play chess or play go or poker, because you can have the system play millions and millions of games against itself and basically fine-tune itself. But it doesn't really work in the real world." The limit, in one image: "If you want to train a car to drive itself, you're not going to do it with reinforcement learning. It's going to crash thousands of times." On robotics he's careful rather than dismissive: "Reinforcement learning can be part of the solution, but it's not the complete answer. It's not sufficient." Third, self-supervised learning. You tell the machine nothing at all. "And this is what has enabled the recent progress in natural language understanding and chatbots." The strange part is that you stop asking for a task: "You don't train the system to accomplish any particular task. You just train it to basically capture the structure..." The method is deliberate sabotage: "You take a piece of text, you corrupt it in some way, by for example removing some words, and then you train a big neural net to predict the words that are missing." And one narrow version of that trick runs every chatbot on Earth: "A special case of this is that you take a piece of text and the last word in that text is not visible, and so you train the system to predict the last word in that text — and this is the way large language models are trained on." So why did the third one win? Supervised learning needs a human. Reinforcement learning needs a crash. Self-supervised learning needs neither — because the missing word and the correct answer are the same thing. The data grades itself.

Big Brain AI

49,293 views • 2 months ago

New groundbreaking research in AI x Robotics! ⦿ The system can autonomously perform various mobile bimanual tasks, including cooking, cleaning, housekeeping, and operating an elevator. ⦿ The hardware costs only $32k. ⦿ Learning is achieved through human demonstration data and co-training.

New groundbreaking research in AI x Robotics! ⦿ The system can autonomously perform various mobile bimanual tasks, including cooking, cleaning, housekeeping, and operating an elevator. ⦿ The hardware costs only $32k. ⦿ Learning is achieved through human demonstration data and co-training.

The Humanoid Hub

10,835 views • 2 years ago

The future of robotics isn’t being built in factories, it’s being trained in simulation MasterBOT’s proprietary AI model is learning in real-time, powered by reinforcement and continuous feedback loops This is how machines learn to think $BOT

The future of robotics isn’t being built in factories, it’s being trained in simulation MasterBOT’s proprietary AI model is learning in real-time, powered by reinforcement and continuous feedback loops This is how machines learn to think $BOT

MasterBOT

51,189 views • 1 year ago

Noble Machines' Moby robot hauling a 50 lb crate. The whole-body control (WBC) is the physical foundation of the learning stack. It absorbs payload variation, contact response, and center-of-mass shifts in real time at the control layer and enabled cleaner data-focused learning

Noble Machines' Moby robot hauling a 50 lb crate. The whole-body control (WBC) is the physical foundation of the learning stack. It absorbs payload variation, contact response, and center-of-mass shifts in real time at the control layer and enabled cleaner data-focused learning

The Humanoid Hub

17,597 views • 5 months ago