How do we make dexterous hands handle both power and precision tasks with ease? 🫳👌🫰 We introduce Power to Precision (💪➡️🎯), our new paper that optimizes both control and fingertip geometry to unlock robust manipulation from power grasp to fine-grained manipulations. With simplified finger motions and augmented fingertips, the... hand can perform diverse motions from pinching a nut🔩 to handling a pan🍳. Check the demos below🎥.show more

The advantage of our method becomes even clearer when compared with the previous kinematics retargeting + original design baseline, both in teleoperation and in the policy trained with the teleoperation data🔥.

We also validate our method in a zero-shot sim-to-real setting (no teleoperation). Check the following uncut continuous grasping videos and the comparison with our previous Dex1B🚀.

Our method jointly optimizes control and fingertip geometry. Simple contact with parallel finger motions makes the resulting actions much more robust. The fingertip geometry is treated as a contact plane and optimized with a neural physics surrogate model ⚡

This project is co-led by I and @laiiwei . Great collaboration with @LuccaChiang , Changwei Jing, @xyz2maureen , and @xiaolonw. Check our project page:

This “Power to Precision” work shows how far we can go when we co-optimize the hand itself and the policy behind it. The gap between power tasks and fine manipulation is closing fast. Good work @jianglong_ye

We will release our mesh & control design tomorrow! 🤗

It is a great journey working on this project, especially we could see dex hand do real dexterous job : )

Interesting stuff. Personally, I think a squishy fingertip with a fingernail would be optimal for picking, similar to our own hands. I don't think anyone has really done that yet!

interesting work!

The co-design approach, which jointly optimizes fingertip geometry and control policy, is a brilliant way to overcome hardware limitations without requiring a complete redesign. What was the core inspiration for the simplified "thumb-index parallel motion" control strategy for precision tasks?

Love this. The thumb–index parallel motion strategy is simple and well aligned with common human precision grasps, and it is also what people often use in teleoperation practice.

This is the future of robot hands, one system that can lift a pan and pinch a nut.

love the work, for the teleoperation do you guys use a webcam or vr? Thanks

We use VR (Vision Pro)

Great work!!! Question: is the extra foam on the finger absolutely necessary?

Thanks! It depends on whether the original design of the hand is suitable for pinch or not. You can look at our comparison video. It is extremely hard for the original design to pinch the tiny nut. Even for larger objects like a marker pen, although it can be grasped with the original hand design, our experiments still show that a flatter contact area provides a more stable grasp.

Very nice 🤠

Good for project