How to chain multiple dexterous skills to tackle complex long-horizon manipulation tasks? Imagine retrieving a LEGO block from a pile, rotating it in-hand, and inserting it at the desired location to build a structure. Introducing our new work - Sequential Dexterity 🧵👇

The core of the system is a learning-based transition feasibility function that progressively finetunes the sub-policies (learned with RL) for enhancing chaining success, which can also be used during skill selection for re-planning from failures and bypassing redundant stages.

Despite being trained only in simulation with a few task objects, our system demonstrates generalization capability to novel object shapes and is able to zero-shot transfer to a real-world robot equipped with a dexterous hand.

We hope Sequential Dexterity paves the path for future research on long-horizon dexterous manipulation. Feel free to check out our code! Website & Paper: Code: Work done w/ Yuanpei Chen, @drfeifei, and Karen Liu at @StanfordAILab.

Congrats, great work!

Thanks Karol!

very cool task and results, congrats!!

Thanks Deepak! Very interested in LEAP hand and functional grasp

super cool work, @chenwang_j!

Thanks Lucy!

Very inspiring, congrats!

Thanks Kevin! huge fan of RoboPianist!