❓I keep coming back to this: what tactile representation does manipulation need? We introduce PanoTac: A dense three-axis Tactile Sensor. In the demo, the robot reorients and inserts a tube, then wiggles the stopper free while the tube sits loosely in the rack. The right side show the tactile... signals that helps robot detect when the stopper releases and when to switch from wiggling to lifting. (1/7)show more

A dense piezoresistive array captures contact geometry, while embedded magnets and magnetometers track fingertip deformation. We fuse these signals into a 12×32×3 tactile field. The visualization on the left shows where contact occurs and how the fingertip is loaded. The sensor is easy to customize and fully open source. (2/7)

Same opaque bottle, two decisions: pour if filled, discard if empty. Lifting the bottle produces shear at the fingertips, providing a cue about its weight. Touch helps the robot make this choice without seeing inside, much like we naturally judge whether a bottle is empty just by lifting it. (3/7)

Our ablations show why combining them matters: understanding how an object sits in the grasp and how it responds during interaction helps the robot decide both how to move and when to change actions. (4/7)

Left: without touch, the robot lifts before the stopper is free. Right: with normal touch only, it keeps wiggling even after the stopper has been released. Shear feedback helps detect release, guiding the robot to stop wiggling and start lifting at the right moment. (5/7)

Human can often tell whether a bottle is empty just by lifting it. PanoTac gives our robot a similar cue. In water pouring experiments, the policy handles unseen liquid colors and even couscous, despite its different density and flow. When vision cannot reveal what’s inside, touch offers a simple, intuitive way to decide what to do: feel the weight. That physical cue remains useful even as appearance and material change. (6/7)

This work is led by @AndreBinglin and @AlexXiaoleJiang , with Zicheng Teng, Xinqi Liu, @YunzhuLiYZ , and @LerrelPinto , and jointly advised by @jeffacce and @binghao_huang. A collaboration between New York University @nyuniversity , City University of New York @CUNY, and Columbia Engineering @CUSEAS. Website: Arxiv:

This is nice! I haven’t read the paper yet, but I’m wondering about extrinsic contact! In general, I’m always curious about sensing designs as they relate to tool use 🙂 excited to see where it goes!

Thanks Rosario! Absolutely! I think extrinsic contact is key to tool use 🙂 In our demo, the change in shear at the fingertips helps the robot feel when the stopper comes free.

@YXWangBot Making a lot of sense to have a vector in tactile

Great work! Can’t wait to see if this sensor can be customized into different shapes

Three-axis feedback could help distinguish slip from contact force during delicate manipulation.

you are so cool

The site lists $9.4 per sensor and up to 100 Hz, while the paper says under $50 excluding readout boards and 60 Hz. Do these reflect a newer revision, or different cost and sampling definitions?

there's an engineer building an open-source tactile world model with zero hardware on it. couldn't get a dm through but the intro's sitting right here