Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Testing the new grasping motion with Microduck: trained in simulation using reinforcement learning, one-shot sim-to-real transfer.

Remi Fabre

5,536 subscribers

13,064 views • 4 days ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

28 Comments

Nick K4 days ago

Cool, is this open-loop? (e.g. no vision input)?

Remi Fabre4 days ago

Remoted controlled here, but I'm working on an autonomous version for simple cases

Nick K4 days ago

How are you doing the remote control though? Does this work on other surfaces (e.g. walls)?

Raphael Gangl4 days ago

how much compute do i need to train it? is my macbook air enoug?

Remi Fabre4 days ago

Might be slow but haven't tested as I use HF jobs all the time

Samantha4 days ago

You do such an awesome job !!!

Phillip Chananda4 days ago

This is great. 👏👏🔥🔥

M1LL54 days ago

Incredible

Helene A4 days ago

Soon he will be playing chess against Reachy mini telling him where to be out his pieces🤣

Remi Fabre4 days ago

Oô

Nikhil Kumar4 days ago

impressive!!

JeCo4 days ago

That's some fine grasping with that large beak! Do your little beta-testers really believe you when you tell them you're actually "working" with microduck? 😄

Remi Fabre4 days ago

They do :)

martin4 days ago

@witcheer

Adam2 days ago

so cool to grow up as a kid with microducks 🥺🔥

dani.3dprint4 days ago

😁

Richard W.4 days ago

The robotics will replace the human.

Lore4 days ago

What did you randomize in simulation for the grasp to transfer, especially contact friction and actuator delay?

Future Visuals4 days ago

Precise motion trained in simulation

Tomorrowist4 days ago

It‘s as interesting as going to church for a one-year-old child to eat.

Lyle Heiss4 days ago

Is it possible to train them without having a great GPU? Would an average laptop suffice?

Remi Fabre4 days ago

As far as I know, you need an NVIDIA GPU to train for real. without one it only runs on CPU, way too slow for a full run. I use HF jobs personnaly

Lyle Heiss4 days ago

Thank you for the reply. Hugging face it is👍

Raphael Gangl4 days ago

does it have similiar sound and speaking capability as reachy?

Remi Fabre4 days ago

it has a small speaker and internet access, so you could use the same software to make Microduck talk. I’m sure a lot of people will do this, but probably won’t because we don’t like the duck talking like a human

Negar Nasiri4 days ago

poor Microduck he is being bullied 😂

someone4 days ago

ofc the man never disappoints wym chess pieces 🥺

Iceland attom3 days ago

NO DUBI; DO NOT PUT THAT ON YOUR PALLAT.

Related Videos

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 views • 1 year ago

The coolest part of Microduck might not be the robot. It's how it learns to move. This repo shows the full reinforcement learning pipeline behind Microduck. The robot learns in simulation using MuJoCo, PPO and GPU-based training, then the trained policy is exported to ONNX and deployed on the real robot. But the real challenge is sim-to-real. The simulation models things like actuator behavior, battery voltage, command delays, friction and even gear backlash. That helps make the learned policy work when it moves from simulation to the physical robot. The repo includes policies for walking, fall recovery, sitting, standing, kicking, ground picking and even roller skating. Train in simulation. Test the behavior. Export the policy. Run it on a real robot.

The coolest part of Microduck might not be the robot. It's how it learns to move. This repo shows the full reinforcement learning pipeline behind Microduck. The robot learns in simulation using MuJoCo, PPO and GPU-based training, then the trained policy is exported to ONNX and deployed on the real robot. But the real challenge is sim-to-real. The simulation models things like actuator behavior, battery voltage, command delays, friction and even gear backlash. That helps make the learned policy work when it moves from simulation to the physical robot. The repo includes policies for walking, fall recovery, sitting, standing, kicking, ground picking and even roller skating. Train in simulation. Test the behavior. Export the policy. Run it on a real robot.

Praveen Kumar Verma

138,017 views • 2 months ago

Using our brain simulator, we’ve trained a reinforcement learning agent to maximize bits per second. Here is the RL policy converting brain data to cursor control in simulation:

Using our brain simulator, we’ve trained a reinforcement learning agent to maximize bits per second. Here is the RL policy converting brain data to cursor control in simulation:

Neuralink

18,410 views • 1 year ago

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.

Humanoids daily

12,391 views • 7 months ago

Pen spinning on WUJI Hand 2, from simulation training to real-world execution. We use motion-reference tracking and reinforcement learning to bring a simulation-trained policy to the physical hand. Code and setup guides are open source for developers and researchers to build on. Code:

Pen spinning on WUJI Hand 2, from simulation training to real-world execution. We use motion-reference tracking and reinforcement learning to bring a simulation-trained policy to the physical hand. Code and setup guides are open source for developers and researchers to build on. Code:

WUJI

36,559 views • 1 month ago

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Abhishek Gupta

10,803 views • 1 year ago

Today, a step forward in open robotics - our results show that sim-to-real zero shot transfer for manipulation is possible. MolmoBot is our open model suite for robotics, trained entirely in simulation on MolmoSpaces.🧵

Today, a step forward in open robotics - our results show that sim-to-real zero shot transfer for manipulation is possible. MolmoBot is our open model suite for robotics, trained entirely in simulation on MolmoSpaces.🧵

Ai2

66,349 views • 7 months ago

Jim Fan says NVIDIA trained humanoid robots to walk and move like humans -- zero-shot transfer from simulation to the real world. 10 years of learning, compressed into 2 hours of simulation. Turns out you don’t need a giant model to master motion. “1.5 million parameters, not billion, to capture the subconscious processing of the human body.” Big embodiment, small brain.

Jim Fan says NVIDIA trained humanoid robots to walk and move like humans -- zero-shot transfer from simulation to the real world. 10 years of learning, compressed into 2 hours of simulation. Turns out you don’t need a giant model to master motion. “1.5 million parameters, not billion, to capture the subconscious processing of the human body.” Big embodiment, small brain.

vitrupo

715,073 views • 1 year ago

Boston Dynamics' Atlas showcases reinforcement learning policies trained using a motion capture suit, developed in partnership with RAI Institute. Both Boston Dynamics and RAI Institute are owned by Hyundai Group.

Boston Dynamics' Atlas showcases reinforcement learning policies trained using a motion capture suit, developed in partnership with RAI Institute. Both Boston Dynamics and RAI Institute are owned by Hyundai Group.

The Humanoid Hub

455,732 views • 1 year ago

Boston Dynamics collaborated with NVIDIA to demonstrate DextrAH-RGB, a workflow for dexterous grasping from stereo RGB input. The end-to-end policy for Atlas robot, trained entirely in NVIDIA Isaac Lab, transfers zero-shot from simulation to the real robot.

Boston Dynamics collaborated with NVIDIA to demonstrate DextrAH-RGB, a workflow for dexterous grasping from stereo RGB input. The end-to-end policy for Atlas robot, trained entirely in NVIDIA Isaac Lab, transfers zero-shot from simulation to the real robot.

The Humanoid Hub

79,781 views • 1 year ago

The future of robotics isn’t being built in factories, it’s being trained in simulation MasterBOT’s proprietary AI model is learning in real-time, powered by reinforcement and continuous feedback loops This is how machines learn to think $BOT

The future of robotics isn’t being built in factories, it’s being trained in simulation MasterBOT’s proprietary AI model is learning in real-time, powered by reinforcement and continuous feedback loops This is how machines learn to think $BOT

MasterBOT

51,189 views • 1 year ago

Simulation provides the ability to unlock autonomy at scale. Our new blog shows how multi-agent reinforcement learning allows us to populate our Wayve Infinity Simulator with the emergent behaviour critical to represent dynamic urban cities in the real world.

Simulation provides the ability to unlock autonomy at scale. Our new blog shows how multi-agent reinforcement learning allows us to populate our Wayve Infinity Simulator with the emergent behaviour critical to represent dynamic urban cities in the real world.

Alex Kendall

66,941 views • 3 years ago

Figure 02 robots are now walking with a smoother gait and increased speed, thanks to sim-to-real reinforcement learning.

Figure 02 robots are now walking with a smoother gait and increased speed, thanks to sim-to-real reinforcement learning.

The Humanoid Hub

146,868 views • 1 year ago

NEW: Disney reveals “ReActor,” a new method for Walt Disney Imagineering’s robotic character pipeline. • Combines reinforcement learning with physics-based simulations to transfer human motion to characters • Could be the next milestone for more lifelike robotic characters

NEW: Disney reveals “ReActor,” a new method for Walt Disney Imagineering’s robotic character pipeline. • Combines reinforcement learning with physics-based simulations to transfer human motion to characters • Could be the next milestone for more lifelike robotic characters

Drew Smith

66,859 views • 6 months ago

i trained a reinforcement learning agent to play diablo 1 in a few hours with pufferlib

i trained a reinforcement learning agent to play diablo 1 in a few hours with pufferlib

Dan Advantage

41,954 views • 11 months ago

Welcome to your new simulation Microduck.

Welcome to your new simulation Microduck.

Directive Creator 🪥

304,039 views • 2 months ago

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

SciTech Era

1,585,680 views • 5 months ago

Trying to teach my new model to run using reinforcement learning (Hyfydy + depRL)

Trying to teach my new model to run using reinforcement learning (Hyfydy + depRL)

Thomas Geijtenbeek

976,585 views • 2 years ago

RoboParty built a humanoid robot in 4 months. Now, the full project is open source. ROBOTO ORIGIN stands about 1.25 m tall, weighs 34 kg and has 23 degrees of freedom. The technical details: • 120 Nm peak torque at the leg joints • 48 V, 15 Ah battery • Mechanical CAD, PCB designs and BOM • Firmware, motor control and ROS 2 deployment • URDF/MJCF models for simulation • Reinforcement learning with Isaac Lab and RSL-RL • Sim-to-sim transfer to MuJoCo and sim-to-real validation The robot can walk, run and jump. What caught my attention is the training to hardware workflow. You can study how learned motion policies move from simulation to a physical humanoid.

RoboParty built a humanoid robot in 4 months. Now, the full project is open source. ROBOTO ORIGIN stands about 1.25 m tall, weighs 34 kg and has 23 degrees of freedom. The technical details: • 120 Nm peak torque at the leg joints • 48 V, 15 Ah battery • Mechanical CAD, PCB designs and BOM • Firmware, motor control and ROS 2 deployment • URDF/MJCF models for simulation • Reinforcement learning with Isaac Lab and RSL-RL • Sim-to-sim transfer to MuJoCo and sim-to-real validation The robot can walk, run and jump. What caught my attention is the training to hardware workflow. You can study how learned motion policies move from simulation to a physical humanoid.

Techniahqrobot | humanoid robots

12,329 views • 11 days ago

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

The TWIML AI Podcast

22,592 views • 10 months ago