How to teach robots to perform long-horizon tasks efficiently and robustly🦾? Introducing MimicPlay - an imitation learning algorithm that uses "cheap human play data". Our approach unlocks both real-time planning through raw perception and strong robustness to disturbances!🧵👇

We observe that human play data is fast and easy to collect, which also covers diverse behavior and situations. On the other hand, although robot data is slow and limited, it does not have embodiment gaps. MimicPlay is a method designed to combine the best of both worlds. (2/N)

MimicPlay is a hierarchical imitation learning algorithm that leverages cheap and non-labeled human play data (10 minutes) for learning the high-level planner and a small amount of robot data (20 demonstrations ~ 20 minutes) for learning a plan-guided low-level controller. (3/N)

The high-level planner is first trained as a goal-conditioned policy. It takes the current and goal images from human play data and outputs a latent plan. We also use a KL-loss to minimize the visual gap between human and robot data. (4/N)

In the second step, we freeze the weights of the trained latent planner. The latent planner takes the current and goal images from the robot data and generates a latent plan to train the low-level controller with a plan-guided imitation learning algorithm. (5/N)

We found MimicPlay significantly outperforms prior methods in performance and sample efficiency. With only 20 robot demonstrations and a planner learned with 10 minutes of human play data (shared across tasks), MimicPlay can perform long-horizon tasks such as baking foods. (6/N)

More importantly, after training multiple tasks within one model, MimicPlay is able to generalize to new tasks with unseen temporal compositions. (7/N)

We further test MimicPlay in more challenging multi-task learning settings, where we found that MimicPlay has the smallest performance drop compared to prior methods. This result highlights MimicPlay's capability to handle diverse tasks within one model. (8/N)

We hope MimicPlay paves the path for future research to scale up robot learning with affordable human costs. 🌐Project site 📄PDF (9/N)

Work done during @NVIDIA internship with @DrJimFan, @JiankaiSun, @RuohanZhang76, @drfeifei, @danfei_xu, @yukez and @AnimaAnandkumar. (in collaboration with @StanfordAILab) (N/N)

Refreshing to see this on my feed, awesome work @chenwang_j !