Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Testing the new grasping motion with Microduck: trained in simulation using reinforcement learning, one-shot sim-to-real transfer.

Remi Fabre

5,536 subscribers

13,064 просмотров • 4 дней назад •via X (Twitter)

Наука и технологии Образование

Anya Rossi• Live Now

Private livecam show

Комментарии: 28

Фото профиля Nick K

Nick K4 дней назад

Cool, is this open-loop? (e.g. no vision input)?

Фото профиля Remi Fabre

Remi Fabre4 дней назад

Remoted controlled here, but I'm working on an autonomous version for simple cases

Фото профиля Nick K

Nick K4 дней назад

How are you doing the remote control though? Does this work on other surfaces (e.g. walls)?

Фото профиля Raphael Gangl

Raphael Gangl4 дней назад

how much compute do i need to train it? is my macbook air enoug?

Фото профиля Remi Fabre

Remi Fabre4 дней назад

Might be slow but haven't tested as I use HF jobs all the time

Фото профиля Samantha

Samantha4 дней назад

You do such an awesome job !!!

Фото профиля Phillip Chananda

Phillip Chananda4 дней назад

This is great. 👏👏🔥🔥

Фото профиля M1LL5

M1LL54 дней назад

Incredible

Фото профиля Helene A

Helene A4 дней назад

Soon he will be playing chess against Reachy mini telling him where to be out his pieces🤣

Фото профиля Remi Fabre

Remi Fabre4 дней назад

Oô

Фото профиля Nikhil Kumar

Nikhil Kumar4 дней назад

impressive!!

Фото профиля JeCo

JeCo4 дней назад

That's some fine grasping with that large beak! Do your little beta-testers really believe you when you tell them you're actually "working" with microduck? 😄

Фото профиля Remi Fabre

Remi Fabre4 дней назад

They do :)

Фото профиля martin

martin4 дней назад

@witcheer

Фото профиля Adam

Adam2 дней назад

so cool to grow up as a kid with microducks 🥺🔥

Фото профиля dani.3dprint

dani.3dprint4 дней назад

😁

Фото профиля Richard W.

Richard W.4 дней назад

The robotics will replace the human.

Фото профиля Lore

Lore4 дней назад

What did you randomize in simulation for the grasp to transfer, especially contact friction and actuator delay?

Фото профиля Future Visuals

Future Visuals4 дней назад

Precise motion trained in simulation

Фото профиля Tomorrowist

Tomorrowist4 дней назад

It‘s as interesting as going to church for a one-year-old child to eat.

Фото профиля Lyle Heiss

Lyle Heiss4 дней назад

Is it possible to train them without having a great GPU? Would an average laptop suffice?

Фото профиля Remi Fabre

Remi Fabre4 дней назад

As far as I know, you need an NVIDIA GPU to train for real. without one it only runs on CPU, way too slow for a full run. I use HF jobs personnaly

Фото профиля Lyle Heiss

Lyle Heiss4 дней назад

Thank you for the reply. Hugging face it is👍

Фото профиля Raphael Gangl

Raphael Gangl4 дней назад

does it have similiar sound and speaking capability as reachy?

Фото профиля Remi Fabre

Remi Fabre4 дней назад

it has a small speaker and internet access, so you could use the same software to make Microduck talk. I’m sure a lot of people will do this, but probably won’t because we don’t like the duck talking like a human

Фото профиля Negar Nasiri

Negar Nasiri4 дней назад

poor Microduck he is being bullied 😂

Фото профиля someone

someone4 дней назад

ofc the man never disappoints wym chess pieces 🥺

Фото профиля Iceland attom

Iceland attom3 дней назад

NO DUBI; DO NOT PUT THAT ON YOUR PALLAT.

Похожие видео

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 просмотров • 1 год назад

The coolest part of Microduck might not be the robot. It's how it learns to move. This repo shows the full reinforcement learning pipeline behind Microduck. The robot learns in simulation using MuJoCo, PPO and GPU-based training, then the trained policy is exported to ONNX and deployed on the real robot. But the real challenge is sim-to-real. The simulation models things like actuator behavior, battery voltage, command delays, friction and even gear backlash. That helps make the learned policy work when it moves from simulation to the physical robot. The repo includes policies for walking, fall recovery, sitting, standing, kicking, ground picking and even roller skating. Train in simulation. Test the behavior. Export the policy. Run it on a real robot.

The coolest part of Microduck might not be the robot. It's how it learns to move. This repo shows the full reinforcement learning pipeline behind Microduck. The robot learns in simulation using MuJoCo, PPO and GPU-based training, then the trained policy is exported to ONNX and deployed on the real robot. But the real challenge is sim-to-real. The simulation models things like actuator behavior, battery voltage, command delays, friction and even gear backlash. That helps make the learned policy work when it moves from simulation to the physical robot. The repo includes policies for walking, fall recovery, sitting, standing, kicking, ground picking and even roller skating. Train in simulation. Test the behavior. Export the policy. Run it on a real robot.

Praveen Kumar Verma

138,017 просмотров • 2 месяцев назад

Using our brain simulator, we’ve trained a reinforcement learning agent to maximize bits per second. Here is the RL policy converting brain data to cursor control in simulation:

Using our brain simulator, we’ve trained a reinforcement learning agent to maximize bits per second. Here is the RL policy converting brain data to cursor control in simulation:

Neuralink

18,410 просмотров • 1 год назад

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.

Humanoids daily

12,391 просмотров • 7 месяцев назад

Pen spinning on WUJI Hand 2, from simulation training to real-world execution. We use motion-reference tracking and reinforcement learning to bring a simulation-trained policy to the physical hand. Code and setup guides are open source for developers and researchers to build on. Code:

Pen spinning on WUJI Hand 2, from simulation training to real-world execution. We use motion-reference tracking and reinforcement learning to bring a simulation-trained policy to the physical hand. Code and setup guides are open source for developers and researchers to build on. Code:

WUJI

36,559 просмотров • 1 месяц назад

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Abhishek Gupta

10,803 просмотров • 1 год назад

Today, a step forward in open robotics - our results show that sim-to-real zero shot transfer for manipulation is possible. MolmoBot is our open model suite for robotics, trained entirely in simulation on MolmoSpaces.🧵

Today, a step forward in open robotics - our results show that sim-to-real zero shot transfer for manipulation is possible. MolmoBot is our open model suite for robotics, trained entirely in simulation on MolmoSpaces.🧵

Ai2

66,349 просмотров • 7 месяцев назад

Jim Fan says NVIDIA trained humanoid robots to walk and move like humans -- zero-shot transfer from simulation to the real world. 10 years of learning, compressed into 2 hours of simulation. Turns out you don’t need a giant model to master motion. “1.5 million parameters, not billion, to capture the subconscious processing of the human body.” Big embodiment, small brain.

Jim Fan says NVIDIA trained humanoid robots to walk and move like humans -- zero-shot transfer from simulation to the real world. 10 years of learning, compressed into 2 hours of simulation. Turns out you don’t need a giant model to master motion. “1.5 million parameters, not billion, to capture the subconscious processing of the human body.” Big embodiment, small brain.

vitrupo

715,073 просмотров • 1 год назад

Boston Dynamics' Atlas showcases reinforcement learning policies trained using a motion capture suit, developed in partnership with RAI Institute. Both Boston Dynamics and RAI Institute are owned by Hyundai Group.

Boston Dynamics' Atlas showcases reinforcement learning policies trained using a motion capture suit, developed in partnership with RAI Institute. Both Boston Dynamics and RAI Institute are owned by Hyundai Group.

The Humanoid Hub

455,732 просмотров • 1 год назад

Boston Dynamics collaborated with NVIDIA to demonstrate DextrAH-RGB, a workflow for dexterous grasping from stereo RGB input. The end-to-end policy for Atlas robot, trained entirely in NVIDIA Isaac Lab, transfers zero-shot from simulation to the real robot.

Boston Dynamics collaborated with NVIDIA to demonstrate DextrAH-RGB, a workflow for dexterous grasping from stereo RGB input. The end-to-end policy for Atlas robot, trained entirely in NVIDIA Isaac Lab, transfers zero-shot from simulation to the real robot.

The Humanoid Hub

79,781 просмотров • 1 год назад

The future of robotics isn’t being built in factories, it’s being trained in simulation MasterBOT’s proprietary AI model is learning in real-time, powered by reinforcement and continuous feedback loops This is how machines learn to think $BOT

The future of robotics isn’t being built in factories, it’s being trained in simulation MasterBOT’s proprietary AI model is learning in real-time, powered by reinforcement and continuous feedback loops This is how machines learn to think $BOT

MasterBOT

51,189 просмотров • 1 год назад

Simulation provides the ability to unlock autonomy at scale. Our new blog shows how multi-agent reinforcement learning allows us to populate our Wayve Infinity Simulator with the emergent behaviour critical to represent dynamic urban cities in the real world.

Simulation provides the ability to unlock autonomy at scale. Our new blog shows how multi-agent reinforcement learning allows us to populate our Wayve Infinity Simulator with the emergent behaviour critical to represent dynamic urban cities in the real world.

Alex Kendall

66,941 просмотров • 3 лет назад

Figure 02 robots are now walking with a smoother gait and increased speed, thanks to sim-to-real reinforcement learning.

Figure 02 robots are now walking with a smoother gait and increased speed, thanks to sim-to-real reinforcement learning.

The Humanoid Hub

146,868 просмотров • 1 год назад

NEW: Disney reveals “ReActor,” a new method for Walt Disney Imagineering’s robotic character pipeline. • Combines reinforcement learning with physics-based simulations to transfer human motion to characters • Could be the next milestone for more lifelike robotic characters

NEW: Disney reveals “ReActor,” a new method for Walt Disney Imagineering’s robotic character pipeline. • Combines reinforcement learning with physics-based simulations to transfer human motion to characters • Could be the next milestone for more lifelike robotic characters

Drew Smith

66,859 просмотров • 6 месяцев назад

i trained a reinforcement learning agent to play diablo 1 in a few hours with pufferlib

i trained a reinforcement learning agent to play diablo 1 in a few hours with pufferlib

Dan Advantage

41,954 просмотров • 11 месяцев назад

Welcome to your new simulation Microduck.

Welcome to your new simulation Microduck.

Directive Creator 🪥

304,039 просмотров • 2 месяцев назад

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

SciTech Era

1,585,680 просмотров • 5 месяцев назад

Trying to teach my new model to run using reinforcement learning (Hyfydy + depRL)

Trying to teach my new model to run using reinforcement learning (Hyfydy + depRL)

Thomas Geijtenbeek

976,585 просмотров • 2 лет назад

RoboParty built a humanoid robot in 4 months. Now, the full project is open source. ROBOTO ORIGIN stands about 1.25 m tall, weighs 34 kg and has 23 degrees of freedom. The technical details: • 120 Nm peak torque at the leg joints • 48 V, 15 Ah battery • Mechanical CAD, PCB designs and BOM • Firmware, motor control and ROS 2 deployment • URDF/MJCF models for simulation • Reinforcement learning with Isaac Lab and RSL-RL • Sim-to-sim transfer to MuJoCo and sim-to-real validation The robot can walk, run and jump. What caught my attention is the training to hardware workflow. You can study how learned motion policies move from simulation to a physical humanoid.

RoboParty built a humanoid robot in 4 months. Now, the full project is open source. ROBOTO ORIGIN stands about 1.25 m tall, weighs 34 kg and has 23 degrees of freedom. The technical details: • 120 Nm peak torque at the leg joints • 48 V, 15 Ah battery • Mechanical CAD, PCB designs and BOM • Firmware, motor control and ROS 2 deployment • URDF/MJCF models for simulation • Reinforcement learning with Isaac Lab and RSL-RL • Sim-to-sim transfer to MuJoCo and sim-to-real validation The robot can walk, run and jump. What caught my attention is the training to hardware workflow. You can study how learned motion policies move from simulation to a physical humanoid.

Techniahqrobot | humanoid robots

12,329 просмотров • 11 дней назад

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

The TWIML AI Podcast

22,592 просмотров • 10 месяцев назад