Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Introduce HumanPlus - Shadowing part Humanoids are born for using human data. We build a real-time shadowing system using a single RGB camera and a whole-body policy for cloning human motion. Examples: - boxing🥊 - playing the piano🎹/ping pong - tossing - typing Open-sourced!

Zipeng Fu

13,252 subscribers

230,117 views • 2 years ago •via X (Twitter)

Gaming Science & Technology Education

Anya Rossi• Live Now

Private livecam show

10 Comments

Zipeng Fu2 years ago

Which hardware platform should HumanPlus be embodied on? We build our own 33-DoF humanoid with two dexterous hands using components: - Inspire-Robots RH56DFX hands - @UnitreeRobotics H1 robot - @ROBOTIS Dynamixel motors - @Razer webcams We open-source our hardware design.

Zipeng Fu2 years ago

Naively copying joints from humans to humanoids does not work due to gravity and different actuations. We train a transformer-based whole-body RL policy in IsaacGym simulation with realistic physics using AMASS dataset containing 40 hours of human motion:

Zipeng Fu2 years ago

To retarget from humans to humanoids, we copy the corresponding Euler angles from SMPL-X to our humanoid model. We use open-sourced SOTA human pose and hand estimation methods (thanks!) - WHAM for body: - HaMeR for hands:

Zipeng Fu2 years ago

Compared with other teleoperation methods, shadowing - is affordable - requires only 1 human operator - avoids singularities - natively supports whole-body control

Zipeng Fu2 years ago

Shadowing is an efficient data collection pipeline. We then perform supervised behavior cloning to train skill policies using egocentric vision, allowing humanoids to complete different tasks autonomously by imitating human skills.

Zipeng Fu2 years ago

This project is not possible without our team of experts, covering from computer graphics to robot learning to robot hardware: - co-leads: @qingqing_zhao_ @Qi_Wu577 - advisors: @chelseabfinn @GordonWetzstein project website: hardware: code:

Yuchen Jin2 years ago

how much can I buy it? I need ten of these buddies shadowing me

Michael Black2 years ago

Nice!

Boris Belousov2 years ago

Very impressive work!

Yi Li2 years ago

very impressive!

Related Videos

Introduce HumanPlus - Autonomous Skills part Humanoids are born for using human data. Imitating humans, our humanoid learns: - fold sweatshirts - unload objects from warehouse racks - diverse locomotion skills (squatting, jumping, standing) - greet another robot Open-sourced!

Introduce HumanPlus - Autonomous Skills part Humanoids are born for using human data. Imitating humans, our humanoid learns: - fold sweatshirts - unload objects from warehouse racks - diverse locomotion skills (squatting, jumping, standing) - greet another robot Open-sourced!

Zipeng Fu

158,140 views • 2 years ago

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 views • 1 year ago

🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform actions like pick and place, walking, kicking, boxing, etc - 💡Unleash the potential of humanoids with human cognitive skills and adaptability 🔗:

🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform actions like pick and place, walking, kicking, boxing, etc - 💡Unleash the potential of humanoids with human cognitive skills and adaptability 🔗:

Tairan He

96,714 views • 2 years ago

🤔 Ever wondered if simulation-based animation/avatar learnings can be applied to real humanoid in real-time? 🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform actions like pick and place, walking, kicking, boxing, etc - 💡Unleash the potential of humanoids with human cognitive skills and adaptability 🔗: 📄: 🎬: H2O proposes: - A scalable retargeting framework for obtaining large-scale humanoid motion dataset, intelligently filtering out infeasible motion for the humanoid embodiment. - We train a full-body motion imitator (similar to PHC) and deploy to the real world zero-shot. - Using this framework, we enable real-time teleoperation of a humanoid via a human operator and webcam, performing skills such as pick and place, kicking, walking strollers, etc. Team: Tairan He Zhengyi “Zen” Luo Wenli Xiao @ChongZitaZhang Kris Kitani Changliu Liu Guanya Shi

Zhengyi “Zen” Luo

47,287 views • 2 years ago

Excited to introduce COLA: Learning Human–Humanoid Coordination for Collaborative Object Carrying 🤝🤖 COLA makes humanoids truly helpful in human collaboration — capable of carrying objects, pushing carts, or responding to human push commands. It provides a proprioception-only policy for compliant human–humanoid coordination across diverse movement patterns. The core idea is simple yet effective: 👉 Fine-tune a collaborative policy from a locomotion policy using a residual teacher. 👉 Train in simulation and distill to a real-world student policy for deployment. Paper: Project:

Excited to introduce COLA: Learning Human–Humanoid Coordination for Collaborative Object Carrying 🤝🤖 COLA makes humanoids truly helpful in human collaboration — capable of carrying objects, pushing carts, or responding to human push commands. It provides a proprioception-only policy for compliant human–humanoid coordination across diverse movement patterns. The core idea is simple yet effective: 👉 Fine-tune a collaborative policy from a locomotion policy using a residual teacher. 👉 Train in simulation and distill to a real-world student policy for deployment. Paper: Project:

Siyuan Huang

21,892 views • 8 months ago

Directional light testing for particle self-shadowing using a volumetric mesh #threejs #tsl #threejsjourney

Directional light testing for particle self-shadowing using a volumetric mesh #threejs #tsl #threejsjourney

MisterPrada

13,360 views • 6 months ago

Excited to introduce TWIST2, our next-generation humanoid data collection system. TWIST2 is portable (use anywhere, no MoCap), scalable (100+ demos in 15 mins), and holistic (unlock major whole-body human skills). Fully open-sourced:

Excited to introduce TWIST2, our next-generation humanoid data collection system. TWIST2 is portable (use anywhere, no MoCap), scalable (100+ demos in 15 mins), and holistic (unlock major whole-body human skills). Fully open-sourced:

Yanjie Ze

100,291 views • 7 months ago

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

The Humanoid Hub

17,395 views • 9 months ago

Not using your humanoid to make humanoids is proof that humanoids are a scam. All commercial applications with scale will be won by super-human, purpose-build robots.

Not using your humanoid to make humanoids is proof that humanoids are a scam. All commercial applications with scale will be won by super-human, purpose-build robots.

Simon Kalouche

147,978 views • 1 year ago

Sharpa’s North robot autonomously playing ping-pong against a human at CES 2026

Sharpa’s North robot autonomously playing ping-pong against a human at CES 2026

The Humanoid Hub

41,334 views • 5 months ago

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

Zhengyi “Zen” Luo

61,889 views • 7 months ago

Introduce OmniH2O, a learning-based system for whole-body humanoid teleop and autonomy: 🦾Robust loco-mani policy 🦸Universal teleop interface: VR, verbal, RGB 🧠Autonomy via ShitposterAI or imitation 🔗Release the first whole-body humanoid dataset

Introduce OmniH2O, a learning-based system for whole-body humanoid teleop and autonomy: 🦾Robust loco-mani policy 🦸Universal teleop interface: VR, verbal, RGB 🧠Autonomy via ShitposterAI or imitation 🔗Release the first whole-body humanoid dataset

Tairan He

129,997 views • 2 years ago

❗HUMAN CLONING IS REAL❗ Soulless Cloned Humans Living Among Us HUMAN CLONES LIVING AMONG US I see a lot of focus on masks used to mimic powerful people, but human clones are rarely mentioned. Human cloning is real and has been going on for quite some time. It's unbelievable, beyond imagination, but it's scientific and a fact of life.

❗HUMAN CLONING IS REAL❗ Soulless Cloned Humans Living Among Us HUMAN CLONES LIVING AMONG US I see a lot of focus on masks used to mimic powerful people, but human clones are rarely mentioned. Human cloning is real and has been going on for quite some time. It's unbelievable, beyond imagination, but it's scientific and a fact of life.

John F. Kennedy Jr Q

1,009,572 views • 4 months ago

NVIDIA has open-sourced SONIC, a humanoid behavior foundation model that gives robots a core set of motor skills learned from large-scale human motion data.

NVIDIA has open-sourced SONIC, a humanoid behavior foundation model that gives robots a core set of motor skills learned from large-scale human motion data.

The Humanoid Hub

33,542 views • 4 months ago

Champ Controllable and Consistent Human Image Animation with 3D Parametric Guidance In this study, we introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework to enhance shape alignment and motion

Champ Controllable and Consistent Human Image Animation with 3D Parametric Guidance In this study, we introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework to enhance shape alignment and motion

AK

194,356 views • 2 years ago

TextOp: A real-time text-to-motion framework for humanoid robots. It allows users to instruct the robot using natural language and to modify commands on the fly, producing smooth, whole-body motions instantly.

TextOp: A real-time text-to-motion framework for humanoid robots. It allows users to instruct the robot using natural language and to modify commands on the fly, producing smooth, whole-body motions instantly.

The Humanoid Hub

25,125 views • 7 months ago

🎾Your humanoid tennis player is here!🤖 Introducing LATENT (Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data) — the world’s first real-time whole-body planning and control algorithm for athletic humanoid tennis. For the first time, a humanoid robot can sustain high-dynamic, long-horizon tennis rallies with millisecond-level reactions, precise ball striking, and natural whole-body motion. This marks a leap from mechanical motion imitation to intelligent, decision-driven athletic interaction. #Galbot #Robotics #Innovation #AI #Technology #Humanoid

🎾Your humanoid tennis player is here!🤖 Introducing LATENT (Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data) — the world’s first real-time whole-body planning and control algorithm for athletic humanoid tennis. For the first time, a humanoid robot can sustain high-dynamic, long-horizon tennis rallies with millisecond-level reactions, precise ball striking, and natural whole-body motion. This marks a leap from mechanical motion imitation to intelligent, decision-driven athletic interaction. #Galbot #Robotics #Innovation #AI #Technology #Humanoid

Galbot

45,552 views • 3 months ago

With Helix 02, Figure has introduced a new foundational layer for whole-body control that replaces 109k lines of hand-engineered C++ with a single neural prior for stable, natural motion. - Learned from 1,000+ hours of human motion data and sim-to-real RL across 200,000+ parallel environments. - A 10M-parameter neural network that takes full-body joint state and base motion as input and outputs joint-level actuator commands at 1 kHz (1,000 times a second).

With Helix 02, Figure has introduced a new foundational layer for whole-body control that replaces 109k lines of hand-engineered C++ with a single neural prior for stable, natural motion. - Learned from 1,000+ hours of human motion data and sim-to-real RL across 200,000+ parallel environments. - A 10M-parameter neural network that takes full-body joint state and base motion as input and outputs joint-level actuator commands at 1 kHz (1,000 times a second).

The Humanoid Hub

28,004 views • 4 months ago

All of these articles on the body area networks and they never bothered to inform us they were using our body are network for the new E health system?? Weird huh? Look at all those diagrams with a human body in them

All of these articles on the body area networks and they never bothered to inform us they were using our body are network for the new E health system?? Weird huh? Look at all those diagrams with a human body in them

Dawn

16,454 views • 1 year ago

CMU researchers, in collaboration with NVIDIA, present ASAP, a two-stage framework for humanoid robot agility. It pre-trains motion policies on human data, then refines them with real-world corrections using a delta action model, which adjusts for simulation mismatches.

CMU researchers, in collaboration with NVIDIA, present ASAP, a two-stage framework for humanoid robot agility. It pre-trains motion policies on human data, then refines them with real-world corrections using a delta action model, which adjusts for simulation mismatches.

The Humanoid Hub

1,826,529 views • 1 year ago