Let’s think about humanoid robots outside carrying the box. How about having the humanoid come out the door, interact with humans, and even dance? Introducing Expressive Whole-Body Control for Humanoid Robots: See how our robot performs rich, diverse, and expressive motions in the real world 👇🧵

The humanoid robot is expressive, it is your friend, it says hi to everyone 👋, hugs you 🤗, and dances with you 💃. (2/n)

The humanoid robot is robust at the same time, and it walks a long way 🏞️. (3/n)

So how do we do it? It is Sim2Real again! We train from simulation using a large-scale motion capture dataset and RL at the same time. We learn a policy that tracks a given reference motion and asks the robot to perform the same. Learning with a lot of motions allows generalization to new unseen motions (from e.g., text2motion, video2motion) in the real world. (4/n)

Of course, Sim2Real is not going to be easy. We find the actual robot has a much smaller DoF (19 DoF with Unitree H1) compared to a humanoid model in a typical graphics paper (69 DoF). The torque limits are also different. This makes all the graphics approaches not directly work. Our design: Make the lower body do robust walking, as robust as possible given a velocity command; Make the upper body do all rich motions with imitation. (5/n)

That is it! Please check our paper for more details: A fantastic collaboration with @xuxin_cheng, @JiYandong, Junming Chen, @RchalYang, @EpisodeYang

Great work! Congratulations!

Thank you Guanya!

Just because humans can do or make something doesn’t mean they should…

Congrats Xiaolong! BTW my nostalgia was triggered by this demo🥹

Come.