Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Sim2Real RL for Vision-Based Dexterous Manipulation on Humanoids TLDR - we train a humanoid robot with two multifingered hands to perform a range of dexterous manipulation tasks robust generalization and high performance without human demonstration :D

Toru

1,780 subscribers

49,425 views • 1 year ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

5 Comments

Chaoyi Pan1 year ago

The result is more impressive considering this robot's upper body doesn’t even have an absolute encoder 🙈

atharva1 year ago

impressive results, like the sample efficient policy learning used.

Tao Huang1 year ago

Impressive work! May I ask what makes humanoid manipulation different from bimanual arm manipulation?

Miguel Guau!1 year ago

¡Qué interesante avance en Sim2Real RL! 🤖 La manipulación dexterous sin demostración humana es un paso enorme hacia la autonomía robótica. ¡Espero ver más aplicaciones en el futuro! 🚀

HumanoidRobots.com1 year ago

great demonstration. thanks for sharing!

Related Videos

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

Mandi Zhao

120,188 views • 1 year ago

Humanoid robot transforms into a Thanksgiving chef and household butler 👩‍🍳 Of course, these close-ups showcase Sahrpa's dexterous hands and highly flexible, precise manipulation abilities, just like a real human.

Humanoid robot transforms into a Thanksgiving chef and household butler 👩‍🍳 Of course, these close-ups showcase Sahrpa's dexterous hands and highly flexible, precise manipulation abilities, just like a real human.

CyberRobo

19,632 views • 6 months ago

Scaling vision-language-action (VLA) models to high-DoF dexterous hands has long been a "holy grail" challenge due to the high-dimensional action space and data scarcity. As a wrap up of the year 2025, we are releasing GR-Dexter, a holistic hardware-model-data framework for generalist manipulation on a bimanual dexterous-hand robot. This is the first VLA system to achieve: ✅ High-DoF Control: Managing a 56-DoF bimanual system (21-DoF per hand). ✅ Long-Horizon Tasks with tool use: Vacuuming, bread serving with tongs, and table decluttering. ✅ Open-World Generalization: Robust performance with unseen objects and abstract instructions. Project page: ArXiv:

Scaling vision-language-action (VLA) models to high-DoF dexterous hands has long been a "holy grail" challenge due to the high-dimensional action space and data scarcity. As a wrap up of the year 2025, we are releasing GR-Dexter, a holistic hardware-model-data framework for generalist manipulation on a bimanual dexterous-hand robot. This is the first VLA system to achieve: ✅ High-DoF Control: Managing a 56-DoF bimanual system (21-DoF per hand). ✅ Long-Horizon Tasks with tool use: Vacuuming, bread serving with tongs, and table decluttering. ✅ Open-World Generalization: Robust performance with unseen objects and abstract instructions. Project page: ArXiv:

Xiao Ma

93,641 views • 5 months ago

This dexterous robotic hand can see, feel, and solve a Rubik’s cube BrainCo's Revo 3 dexterous robotic hand, a Chinese-developed system with integrated vision, full-palm tactile sensors, and high-precision actuators for complex manipulation tasks

This dexterous robotic hand can see, feel, and solve a Rubik’s cube BrainCo's Revo 3 dexterous robotic hand, a Chinese-developed system with integrated vision, full-palm tactile sensors, and high-precision actuators for complex manipulation tasks

Science girl

19,923 views • 2 months ago

ViTacFormer is a unified visuo-tactile framework for dexterous robot manipulation. It fuses high-res visual+tactile data using cross-attention and predicts future tactile signals via an autoregressive head, enabling multi-fingered hands to perform precise, long-horizon tasks.

ViTacFormer is a unified visuo-tactile framework for dexterous robot manipulation. It fuses high-res visual+tactile data using cross-attention and predicts future tactile signals via an autoregressive head, enabling multi-fingered hands to perform precise, long-horizon tasks.

The Humanoid Hub

10,816 views • 11 months ago

Another day, another humanoid robot from china AGIBOT introduces GO-1, a generalist foundation model that integrates a vision-language model with a latent planner for enhanced long-horizon and dexterous manipulation.

Another day, another humanoid robot from china AGIBOT introduces GO-1, a generalist foundation model that integrates a vision-language model with a latent planner for enhanced long-horizon and dexterous manipulation.

Chubby♨️

23,513 views • 1 year ago

Eric Jang weighs in on RL vs. supervised learning for humanoid robot manipulation tasks.

Eric Jang weighs in on RL vs. supervised learning for humanoid robot manipulation tasks.

The Humanoid Hub

10,656 views • 10 months ago

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:

Haoran Geng

94,717 views • 11 months ago

🔥 #ICRA2026 Best Paper Finalist The era of "robot VLA = single-arm gripper" is ending. Introducing Dexora — the first open-source Vision-Language-Action system for dual-arm, dual-hand, 36-DoF dexterous manipulation. 🦾 Dual Arms 🖐️ Dual Hands 🎯 36 DoF Control 🌍 Open Source Trained on: • 100K simulated trajectories • 10K real-world demonstrations Dexora achieves: ✓ 90%+ success on basic manipulation ✓ Strong dexterous manipulation performance ✓ Cross-embodiment generalization Our key hypothesis: Train on the hardest embodiment. Transfer to simpler robots later. Instead of scaling up gripper policies, we train directly in the most expressive action space and project downward to simpler embodiments. This may be a practical path toward universal robot controllers. 🎥 Demos: 📄 Paper:

🔥 #ICRA2026 Best Paper Finalist The era of "robot VLA = single-arm gripper" is ending. Introducing Dexora — the first open-source Vision-Language-Action system for dual-arm, dual-hand, 36-DoF dexterous manipulation. 🦾 Dual Arms 🖐️ Dual Hands 🎯 36 DoF Control 🌍 Open Source Trained on: • 100K simulated trajectories • 10K real-world demonstrations Dexora achieves: ✓ 90%+ success on basic manipulation ✓ Strong dexterous manipulation performance ✓ Cross-embodiment generalization Our key hypothesis: Train on the hardest embodiment. Transfer to simpler robots later. Instead of scaling up gripper policies, we train directly in the most expressive action space and project downward to simpler embodiments. This may be a practical path toward universal robot controllers. 🎥 Demos: 📄 Paper:

Hao Zhao

16,027 views • 6 days ago

Tactile feedback is one of the most important modalities in manipulation, but has been underutilized in dexterous hands. T-Dex is a framework for learning dexterous policies from tactile play data, beating vision and torque-based methods by 1.7x. 🧵👇

Tactile feedback is one of the most important modalities in manipulation, but has been underutilized in dexterous hands. T-Dex is a framework for learning dexterous policies from tactile play data, beating vision and torque-based methods by 1.7x. 🧵👇

Lerrel Pinto

84,258 views • 3 years ago

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

Chris Paxton

20,486 views • 1 year ago

Humanoid Long Jump. The G1 humanoid from Unitree Robotics has completed a long jump of 1.4 meters - possibly the longest jump ever achieved by a humanoid robot of its size in the world, standing only 1.32 meters tall. The G1 has up to 43 joint motors and a dexterous three-fingered hand, designed for precise manipulation and AI research. Priced from $16,000, it features robust Sim2Real capabilities, making it a compelling choice for robotics developers.

Humanoid Long Jump. The G1 humanoid from Unitree Robotics has completed a long jump of 1.4 meters - possibly the longest jump ever achieved by a humanoid robot of its size in the world, standing only 1.32 meters tall. The G1 has up to 43 joint motors and a dexterous three-fingered hand, designed for precise manipulation and AI research. Priced from $16,000, it features robust Sim2Real capabilities, making it a compelling choice for robotics developers.

Circuit

26,485 views • 1 year ago

San Mateo, California–based Generalist showcases their real-time, end-to-end neural network that controls robots at 100Hz for dexterous manipulation tasks. The model handles fine-motor control, bi-manual coordination, tool use, and generalization across arms and environments.

San Mateo, California–based Generalist showcases their real-time, end-to-end neural network that controls robots at 100Hz for dexterous manipulation tasks. The model handles fine-motor control, bi-manual coordination, tool use, and generalization across arms and environments.

The Humanoid Hub

42,524 views • 1 year ago

“Dexterous capability is directly proportional to the size of the addressable market for general-purpose humanoid robots,” says James Wells, interim CEO and CCO. Sanctuary AI announces its latest technical milestone using hydraulic actuation, opening up a wide range of industrial and high value work tasks that require fine manipulation. #AI #Robotics

“Dexterous capability is directly proportional to the size of the addressable market for general-purpose humanoid robots,” says James Wells, interim CEO and CCO. Sanctuary AI announces its latest technical milestone using hydraulic actuation, opening up a wide range of industrial and high value work tasks that require fine manipulation. #AI #Robotics

Sanctuary AI

16,583 views • 1 year ago

Happy to announce mimic-one: a Scalable Model Recipe for General Purpose Robot Dexterity, the culmination of years of research work in dexterous manipulation with imitation learning.

Happy to announce mimic-one: a Scalable Model Recipe for General Purpose Robot Dexterity, the culmination of years of research work in dexterous manipulation with imitation learning.

Elvis Nava

20,232 views • 1 year ago

Our newest model, π0.7, has some interesting emergent capabilities: it can control a new robot to fold shirts for which we had no shirt folding data, figure out how to use an appliance with language-based coaching, and perform a wide range of dexterous tasks all in one model!

Our newest model, π0.7, has some interesting emergent capabilities: it can control a new robot to fold shirts for which we had no shirt folding data, figure out how to use an appliance with language-based coaching, and perform a wide range of dexterous tasks all in one model!

Physical Intelligence

451,694 views • 1 month ago

📢📢📢 Excited to release ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR25). 🤏🤙✌️With ManipTrans, we can transfer dexterous manipulation skills into robotic hands in simulation and deploy them on a real robot, using a residual policy learned for dex manipulation. 🤖🤖🤖The video below illustrates how the MoCap data can be transferred to Inspire, Shadow, Xhand, Allegro, and Mano. With ManipTrans, we can scale up dex manip data greatly with minimal effort. For more details, please check our -webpage: -code: -huggingface:

📢📢📢 Excited to release ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR25). 🤏🤙✌️With ManipTrans, we can transfer dexterous manipulation skills into robotic hands in simulation and deploy them on a real robot, using a residual policy learned for dex manipulation. 🤖🤖🤖The video below illustrates how the MoCap data can be transferred to Inspire, Shadow, Xhand, Allegro, and Mano. With ManipTrans, we can scale up dex manip data greatly with minimal effort. For more details, please check our -webpage: -code: -huggingface:

Siyuan Huang

20,918 views • 1 year ago

Advancing dexterous manipulation through scalable visual sim-to-real transfer. We are excited to share our RSS paper, “ViserDex: Visual Sim-to-Real for Robust Dexterous In-hand Reorientation.” 🌐 Project page: 1/N 🧵

Advancing dexterous manipulation through scalable visual sim-to-real transfer. We are excited to share our RSS paper, “ViserDex: Visual Sim-to-Real for Robust Dexterous In-hand Reorientation.” 🌐 Project page: 1/N 🧵

Robotic Systems Lab

39,143 views • 25 days ago