Tired of endlessly teleoperating your robot in order to train it? Introducing SkillMimicGen, a data generation system that automatically scales robot imitation learning by synthesizing demos through integrating motion planning and demo adaptation. 1/

SkillMimicGen (SkillGen) takes a small number (~10) of human-teleoperated demos and automatically generates more. Unlike prior work (MimicGen), SkillGen isolates and adapts just demo skill segments in order to focus learning. 2/

SkillGen automates dataset generation by interleaving contact-rich demonstration adaptation with robot motion planning. This enables efficient data generation and policy learning for long-horizon tasks, which MimicGen struggles with. 3/

The skill-based formulation and use of motion planning offer key benefits over MimicGen. For example, SkillGen can generate demos in diverse environments with both large pose variation and even clutter. 4/

SkillGen datasets train high-performing behavioral cloning agents capable of handling diverse initial conditions. These agents are on average 46% more successful than those trained with MimicGen datasets. 5/

SkillGen produces comparable results to the painful alternative of collecting more human demonstrations, and agent performance improves by just generating more SkillGen data! 6/

SkillMimicGen (SkillGen) vastly outperforms MimicGen, especially 1) on tasks with large amounts of pose variation, 2) in cluttered settings, and 3) in real-world deployments. 7/

SkillGen generation is robust to real-world cluttered environments, and SkillGen agents are robust to large pose variations in the real world. 8/

SkillGen also enables zero-shot sim2real transfer! Here, we collected 1 human demo in simulation and trained an agent using SkillGen data, which achieved zero-shot generalization to this long-horizon nut assembly task. 9/

This work was made possible by an amazing team! @CaelanGarrett (co-lead), @bowenwen_me, and Dieter Fox Paper: Website: 10/

@chris_j_paxton But if you can already generate successful demonstrations, why bother to do robot training afterwards?