Teaching bimanual robot hands to perform very complex tasks has been notoriously challenging. In our work, Bidex: Bimanual Dexterity for Complex Tasks, we’ve developed a low-cost system that completes a wide range of highly dexterous tasks in real-time.

Bidex uses a teacher arm connected to a motion capture glove worn by a teleoperator. The teacher arm data is mirrored to any standard robot arm and the motion capture glove data is retargeted to our 21DOF LEAP Hand. This setup is lightweight, portable, precise, and user-friendly.

Bidex can perform many tasks from peeling a cucumber to spraying of cleaning liquids to mobile tasks such as picking up trash. Compared to the vision-based Apple Vision Pro, we find in our user study that Bidex is faster, more intuitive and leads to higher success rates.

Thanks to Bidex's advanced capabilities, collecting high-quality data for behavior cloning is very efficient. The teleoperation data is smoother which results in better policies using less data. We pre-train these policies on internet videos to aid in generalization.

The complete Bidex, which includes two robot arms ($16k), teacher arms ($500), robot hands ($6k), and gloves ($6k), costs under $30,000. We use LEAP Hand v2, a 21 DOF hybrid rigid-soft hand that is compliant yet very strong, easy to assemble, and has human-like kinematics.

The robot hands and Manus glove code are open sourced at Bidex works with many robot arms! #Corl2024 @ManusMeta It was led by me and @yulongli42 with @Jiahui_Yang6709 @mohansrirama Ray Liu, @Haoyu_Xiong_ and advisors @mendonca_rl and @pathak2206

@yulongli42 @Jiahui_Yang6709 @mohansrirama @Haoyu_Xiong_ @mendonca_rl @pathak2206 Bidex is portable so it is easy to teleoperate from anywhere! In future work, we are looking at highly dexterous mobile tasks in-the-wild. Here is a quick preview. :)

Congrats on the launch!

Phenomenal work! 🤖👐 Teaching bimanual robot hands to handle complex tasks is a breakthrough for robotics. Real-time dexterity has the potential to revolutionize industries, from healthcare to waste management. At Social Hardware, we’re focusing on tele-operational systems for hazardous tasks, ensuring both precision and worker safety. It's inspiring to see advancements like Bidex contributing to the evolution of robotics! Looking forward to seeing more. 🚀 #Robotics #TeleOperation #Innovation

Awesome! How do you transfer across different hands? I guess that would be quite challenging.

Yes, we released this code on our website. In short, we optimize for matching pinch grasp similarity between the human and robot hands and optimize for fingertip positions. The concept is based our older works such as Robotic Telekinesis.