正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Testing the new grasping motion with Microduck: trained in simulation using reinforcement learning, one-shot sim-to-real transfer.

Remi Fabre

5,536 subscribers

13,064 次观看 • 4 天前 •via X (Twitter)

科学技术教育

Anya Rossi• Live Now

Private livecam show

28 条评论

Nick K 的头像

Nick K4 天前

Cool, is this open-loop? (e.g. no vision input)?

Remi Fabre 的头像

Remi Fabre4 天前

Remoted controlled here, but I'm working on an autonomous version for simple cases

Nick K 的头像

Nick K4 天前

How are you doing the remote control though? Does this work on other surfaces (e.g. walls)?

Raphael Gangl 的头像

Raphael Gangl4 天前

how much compute do i need to train it? is my macbook air enoug?

Remi Fabre 的头像

Remi Fabre4 天前

Might be slow but haven't tested as I use HF jobs all the time

Samantha 的头像

Samantha4 天前

You do such an awesome job !!!

Phillip Chananda 的头像

Phillip Chananda4 天前

This is great. 👏👏🔥🔥

M1LL5 的头像

M1LL54 天前

Incredible

Helene A 的头像

Helene A4 天前

Soon he will be playing chess against Reachy mini telling him where to be out his pieces🤣

Remi Fabre 的头像

Remi Fabre4 天前

Oô

Nikhil Kumar 的头像

Nikhil Kumar4 天前

impressive!!

JeCo 的头像

JeCo4 天前

That's some fine grasping with that large beak! Do your little beta-testers really believe you when you tell them you're actually "working" with microduck? 😄

Remi Fabre 的头像

Remi Fabre4 天前

They do :)

martin 的头像

martin4 天前

@witcheer

Adam 的头像

Adam2 天前

so cool to grow up as a kid with microducks 🥺🔥

dani.3dprint 的头像

dani.3dprint4 天前

😁

Richard W. 的头像

Richard W.4 天前

The robotics will replace the human.

Lore 的头像

Lore4 天前

What did you randomize in simulation for the grasp to transfer, especially contact friction and actuator delay?

Future Visuals 的头像

Future Visuals3 天前

Precise motion trained in simulation

Tomorrowist 的头像

Tomorrowist4 天前

It‘s as interesting as going to church for a one-year-old child to eat.

Lyle Heiss 的头像

Lyle Heiss4 天前

Is it possible to train them without having a great GPU? Would an average laptop suffice?

Remi Fabre 的头像

Remi Fabre4 天前

As far as I know, you need an NVIDIA GPU to train for real. without one it only runs on CPU, way too slow for a full run. I use HF jobs personnaly

Lyle Heiss 的头像

Lyle Heiss4 天前

Thank you for the reply. Hugging face it is👍

Raphael Gangl 的头像

Raphael Gangl4 天前

does it have similiar sound and speaking capability as reachy?

Remi Fabre 的头像

Remi Fabre4 天前

it has a small speaker and internet access, so you could use the same software to make Microduck talk. I’m sure a lot of people will do this, but probably won’t because we don’t like the duck talking like a human

Negar Nasiri 的头像

Negar Nasiri4 天前

poor Microduck he is being bullied 😂

someone 的头像

someone4 天前

ofc the man never disappoints wym chess pieces 🥺

Iceland attom 的头像

Iceland attom3 天前

NO DUBI; DO NOT PUT THAT ON YOUR PALLAT.

相关视频

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 次观看 • 1 年前

The coolest part of Microduck might not be the robot. It's how it learns to move. This repo shows the full reinforcement learning pipeline behind Microduck. The robot learns in simulation using MuJoCo, PPO and GPU-based training, then the trained policy is exported to ONNX and deployed on the real robot. But the real challenge is sim-to-real. The simulation models things like actuator behavior, battery voltage, command delays, friction and even gear backlash. That helps make the learned policy work when it moves from simulation to the physical robot. The repo includes policies for walking, fall recovery, sitting, standing, kicking, ground picking and even roller skating. Train in simulation. Test the behavior. Export the policy. Run it on a real robot.

The coolest part of Microduck might not be the robot. It's how it learns to move. This repo shows the full reinforcement learning pipeline behind Microduck. The robot learns in simulation using MuJoCo, PPO and GPU-based training, then the trained policy is exported to ONNX and deployed on the real robot. But the real challenge is sim-to-real. The simulation models things like actuator behavior, battery voltage, command delays, friction and even gear backlash. That helps make the learned policy work when it moves from simulation to the physical robot. The repo includes policies for walking, fall recovery, sitting, standing, kicking, ground picking and even roller skating. Train in simulation. Test the behavior. Export the policy. Run it on a real robot.

Praveen Kumar Verma

138,017 次观看 • 2 个月前

Using our brain simulator, we’ve trained a reinforcement learning agent to maximize bits per second. Here is the RL policy converting brain data to cursor control in simulation:

Using our brain simulator, we’ve trained a reinforcement learning agent to maximize bits per second. Here is the RL policy converting brain data to cursor control in simulation:

Neuralink

18,410 次观看 • 1 年前

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.

Humanoids daily

12,391 次观看 • 7 个月前

Pen spinning on WUJI Hand 2, from simulation training to real-world execution. We use motion-reference tracking and reinforcement learning to bring a simulation-trained policy to the physical hand. Code and setup guides are open source for developers and researchers to build on. Code:

Pen spinning on WUJI Hand 2, from simulation training to real-world execution. We use motion-reference tracking and reinforcement learning to bring a simulation-trained policy to the physical hand. Code and setup guides are open source for developers and researchers to build on. Code:

WUJI

36,559 次观看 • 1 个月前

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Abhishek Gupta

10,803 次观看 • 1 年前

Today, a step forward in open robotics - our results show that sim-to-real zero shot transfer for manipulation is possible. MolmoBot is our open model suite for robotics, trained entirely in simulation on MolmoSpaces.🧵

Today, a step forward in open robotics - our results show that sim-to-real zero shot transfer for manipulation is possible. MolmoBot is our open model suite for robotics, trained entirely in simulation on MolmoSpaces.🧵

Ai2

66,349 次观看 • 7 个月前

Jim Fan says NVIDIA trained humanoid robots to walk and move like humans -- zero-shot transfer from simulation to the real world. 10 years of learning, compressed into 2 hours of simulation. Turns out you don’t need a giant model to master motion. “1.5 million parameters, not billion, to capture the subconscious processing of the human body.” Big embodiment, small brain.

Jim Fan says NVIDIA trained humanoid robots to walk and move like humans -- zero-shot transfer from simulation to the real world. 10 years of learning, compressed into 2 hours of simulation. Turns out you don’t need a giant model to master motion. “1.5 million parameters, not billion, to capture the subconscious processing of the human body.” Big embodiment, small brain.

vitrupo

715,073 次观看 • 1 年前

Boston Dynamics' Atlas showcases reinforcement learning policies trained using a motion capture suit, developed in partnership with RAI Institute. Both Boston Dynamics and RAI Institute are owned by Hyundai Group.

Boston Dynamics' Atlas showcases reinforcement learning policies trained using a motion capture suit, developed in partnership with RAI Institute. Both Boston Dynamics and RAI Institute are owned by Hyundai Group.

The Humanoid Hub

455,732 次观看 • 1 年前

Boston Dynamics collaborated with NVIDIA to demonstrate DextrAH-RGB, a workflow for dexterous grasping from stereo RGB input. The end-to-end policy for Atlas robot, trained entirely in NVIDIA Isaac Lab, transfers zero-shot from simulation to the real robot.

Boston Dynamics collaborated with NVIDIA to demonstrate DextrAH-RGB, a workflow for dexterous grasping from stereo RGB input. The end-to-end policy for Atlas robot, trained entirely in NVIDIA Isaac Lab, transfers zero-shot from simulation to the real robot.

The Humanoid Hub

79,781 次观看 • 1 年前

The future of robotics isn’t being built in factories, it’s being trained in simulation MasterBOT’s proprietary AI model is learning in real-time, powered by reinforcement and continuous feedback loops This is how machines learn to think $BOT

The future of robotics isn’t being built in factories, it’s being trained in simulation MasterBOT’s proprietary AI model is learning in real-time, powered by reinforcement and continuous feedback loops This is how machines learn to think $BOT

MasterBOT

51,189 次观看 • 1 年前

Simulation provides the ability to unlock autonomy at scale. Our new blog shows how multi-agent reinforcement learning allows us to populate our Wayve Infinity Simulator with the emergent behaviour critical to represent dynamic urban cities in the real world.

Simulation provides the ability to unlock autonomy at scale. Our new blog shows how multi-agent reinforcement learning allows us to populate our Wayve Infinity Simulator with the emergent behaviour critical to represent dynamic urban cities in the real world.

Alex Kendall

66,941 次观看 • 3 年前

Figure 02 robots are now walking with a smoother gait and increased speed, thanks to sim-to-real reinforcement learning.

Figure 02 robots are now walking with a smoother gait and increased speed, thanks to sim-to-real reinforcement learning.

The Humanoid Hub

146,868 次观看 • 1 年前

NEW: Disney reveals “ReActor,” a new method for Walt Disney Imagineering’s robotic character pipeline. • Combines reinforcement learning with physics-based simulations to transfer human motion to characters • Could be the next milestone for more lifelike robotic characters

NEW: Disney reveals “ReActor,” a new method for Walt Disney Imagineering’s robotic character pipeline. • Combines reinforcement learning with physics-based simulations to transfer human motion to characters • Could be the next milestone for more lifelike robotic characters

Drew Smith

66,859 次观看 • 6 个月前

i trained a reinforcement learning agent to play diablo 1 in a few hours with pufferlib

i trained a reinforcement learning agent to play diablo 1 in a few hours with pufferlib

Dan Advantage

41,954 次观看 • 11 个月前

Welcome to your new simulation Microduck.

Welcome to your new simulation Microduck.

Directive Creator 🪥

304,039 次观看 • 2 个月前

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

SciTech Era

1,585,680 次观看 • 5 个月前

Trying to teach my new model to run using reinforcement learning (Hyfydy + depRL)

Trying to teach my new model to run using reinforcement learning (Hyfydy + depRL)

Thomas Geijtenbeek

976,585 次观看 • 2 年前

RoboParty built a humanoid robot in 4 months. Now, the full project is open source. ROBOTO ORIGIN stands about 1.25 m tall, weighs 34 kg and has 23 degrees of freedom. The technical details: • 120 Nm peak torque at the leg joints • 48 V, 15 Ah battery • Mechanical CAD, PCB designs and BOM • Firmware, motor control and ROS 2 deployment • URDF/MJCF models for simulation • Reinforcement learning with Isaac Lab and RSL-RL • Sim-to-sim transfer to MuJoCo and sim-to-real validation The robot can walk, run and jump. What caught my attention is the training to hardware workflow. You can study how learned motion policies move from simulation to a physical humanoid.

RoboParty built a humanoid robot in 4 months. Now, the full project is open source. ROBOTO ORIGIN stands about 1.25 m tall, weighs 34 kg and has 23 degrees of freedom. The technical details: • 120 Nm peak torque at the leg joints • 48 V, 15 Ah battery • Mechanical CAD, PCB designs and BOM • Firmware, motor control and ROS 2 deployment • URDF/MJCF models for simulation • Reinforcement learning with Isaac Lab and RSL-RL • Sim-to-sim transfer to MuJoCo and sim-to-real validation The robot can walk, run and jump. What caught my attention is the training to hardware workflow. You can study how learned motion policies move from simulation to a physical humanoid.

Techniahqrobot | humanoid robots

12,329 次观看 • 11 天前

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

The TWIML AI Podcast

22,592 次观看 • 10 个月前