Skip navigation
Introduce HumanPlus - Shadowing part
0:00 / 0:53

Introduce HumanPlus - Shadowing part

Zipeng Fu

@zipengfu15K followers

Visit on X
732 likesDownloadOriginal post

230,822 views

Introduce HumanPlus - Shadowing part Humanoids are born for using human data. We build a real-time shadowing system using a single RGB camera and a whole-body policy for cloning human motion. Examples: - boxing🥊 - playing the piano🎹/ping pong - tossing - typing Open-sourced!

10 Comments

Comments from the original post on X (Twitter)

  • Zipeng Fu's profile picture

    @zipengfu

    Which hardware platform should HumanPlus be embodied on? We build our own 33-DoF humanoid with two dexterous hands using components: - Inspire-Robots RH56DFX hands - @UnitreeRobotics H1 robot - @ROBOTIS Dynamixel motors - @Razer webcams We open-source our hardware design.

    48 likes

  • Zipeng Fu's profile picture

    @zipengfu

    Naively copying joints from humans to humanoids does not work due to gravity and different actuations. We train a transformer-based whole-body RL policy in IsaacGym simulation with realistic physics using AMASS dataset containing 40 hours of human motion:

    29 likes

  • Zipeng Fu's profile picture

    @zipengfu

    To retarget from humans to humanoids, we copy the corresponding Euler angles from SMPL-X to our humanoid model. We use open-sourced SOTA human pose and hand estimation methods (thanks!) - WHAM for body: - HaMeR for hands:

    44 likes

  • Zipeng Fu's profile picture

    @zipengfu

    Compared with other teleoperation methods, shadowing - is affordable - requires only 1 human operator - avoids singularities - natively supports whole-body control

    11 likes

  • Zipeng Fu's profile picture

    @zipengfu

    Shadowing is an efficient data collection pipeline. We then perform supervised behavior cloning to train skill policies using egocentric vision, allowing humanoids to complete different tasks autonomously by imitating human skills.

    23 likes

  • Zipeng Fu's profile picture

    @zipengfu

    This project is not possible without our team of experts, covering from computer graphics to robot learning to robot hardware: - co-leads: @qingqing_zhao_ @Qi_Wu577 - advisors: @chelseabfinn @GordonWetzstein project website: hardware: code:

    42 likes

  • Yuchen Jin's profile picture

    @Yuchenj_UW

    how much can I buy it? I need ten of these buddies shadowing me

    3 likes

  • Michael Black's profile picture

    @Michael_J_Black

    Nice!

    1 like

  • Boris Belousov's profile picture

    @_bbelousov

    Very impressive work!

    1 like

  • Yi Li's profile picture

    @Yi__Li

    very impressive!

    1 like