Loading video...

Video Failed to Load

Go Home

ViTacFormer is a unified visuo-tactile framework for dexterous robot manipulation. It fuses high-res visual+tactile data using cross-attention and predicts future tactile signals via an autoregressive head, enabling multi-fingered hands to perform precise, long-horizon tasks.

10,816 views • 1 year ago •via X (Twitter)

7 Comments

The Humanoid Hub's profile picture
The Humanoid Hub1 year ago

More details in this thread:

Mobile Scanner's profile picture
Mobile Scanner1 year ago

Scan any documents, convert images into text, PDF files, etc. 👍

J⏩'s profile picture
J⏩1 year ago

Whenever I see these posts now it's always depressing how sterile and perfect and not real world at all the test environment needs to be. We have come almost nowhere in years.

JennL's profile picture
JennL1 year ago

Edward scissor hands looking bot 🤖

Brian Bellia's profile picture
Brian Bellia1 year ago

I'm losing faith in the general-purpose humanoid promise. Let's face it, it's going to be a decade (at least) before the average person could trust one of these things in their kitchen to prepare a meal. Alternatively, bot buddies could be available in less than half of that time

10turtle's profile picture
10turtle1 year ago

Robots that see and feel ViTacFormer is a big step toward true human-like precision in manipulation.

C12s's profile picture
C12s1 year ago

Merging sight and touch — this is a major leap toward truly human-like robotic dexterity.

Related Videos

I was really impressed by the UMI gripper (Cheng Chi et al.), but a key limitation is that **force-related data wasn’t captured**: humans feel haptic feedback through the mechanical springs, but the robot couldn’t leverage that info, limiting the data’s value for fine-grained manipulation tasks. Led by my amazing students Yolanda Zhu and Binghao Huang, we designed a **portable visuo-tactile gripper** by integrating our dense, flexible tactile arrays with the UMI gripper to enable large-scale in-the-wild data collection. 🔗 We demonstrate **cross-modal representation learning** and **downstream policy learning** on tasks requiring in-hand state estimation (e.g., test tube reorientation) and fine-grained force sensing (e.g., pipette fluid transfer). Key takeaways: - Our flexible tactile arrays store the rich haptic information humans perceive as dense tactile signals. - Portability and robustness are key for in-the-wild data collection; our portable gripper is compact, lightweight, and durable. - Touch provides precise, robust measurements of in-hand object pose, invariant to lighting and viewpoint. - Cross-modal pretraining on large-scale in-the-wild data significantly improves policy robustness and sample efficiency (as shown many times before — and verified again here!). Also check out our previous investigations of dense, flexible tactile grids for understanding human-robot-environment interactions: - Dense tactile glove (Nature ’19): - 3D-ViTac (CoRL ’24):

Yunzhu Li

13,188 views • 11 months ago