正在加载视频...

视频加载失败

返回首页

Contact does not always deform surfaces, especially when interacting with liquids and soft objects. However, most existing tactile sensors rely on surface deformation to infer contact. Excited to share LightTact, a visual-tactile fingertip sensor that makes contact directly visible. It provides: - Deformation-Independent Sensing: contact is detected optically, not...

changyi_lin1's profile picture

Changyi Lin

1,055 subscribers

65,715 次观看 • 4 个月前 •via X (Twitter)

科学技术教育
LIVE

Anya Rossi• Live Now

Private livecam show

9 条评论

Al Bert 的头像
Al Bert4 个月前

how is this different than gelsight?

Kurt Wolf 的头像
Kurt Wolf4 个月前

Does it still work when the lights are off?

Nick Parker 的头像
Nick Parker4 个月前

Very cool trick!! Have you done any sim/search ish work yet to find other front optic shapes that are compatible with this? It’s a bit incompatible with meaningful surface deformation in fact right?

Nick CleanCode 的头像
Nick CleanCode4 个月前

Really cool sensor! Sensing contact with water and cream without deformation... that's genuinely impressive. Makes me wonder where this leaves manufacturing workers though.

Priya Sen 的头像
Priya Sen4 个月前

this is so cool, i need this to stop me from crushing my delicate propagation roots

Michelle Lovato 的头像
Michelle Lovato4 个月前

this is awesome!!!

AI Mastery Guide 的头像
AI Mastery Guide4 个月前

Detecting contact optically instead of relying on surface deformation makes a lot of sense for liquids and soft objects that barely deform at all.

Darren Dawson 的头像
Darren Dawson4 个月前

FITR is fine but what happens when the screen is scratched over time?

Frame 的头像
Frame4 个月前

optical contact sensing is smart. We like the portability angle for multi embodiment.

相关视频

Seeing touch without pressing down! 🫆 Most tactile sensors rely on deformation. Press down, deform the sensor, measure the deformation. But light contact, touching water, cream, soft films, doesn't deform anything. The sensor sees nothing. LightTact flips the approach. It uses optical detection instead. An ambient-blocking configuration suppresses external light and internal illumination everywhere except at actual contact points. Only scattered light from true contact gets through. The results? High-contrast raw images. Non-contact pixels stay near-black. Contact pixels show the natural appearance of whatever's touching. This works across material properties, contact forces, surface colors, and lighting conditions. Water, facial cream, thin films, liquids, all become visible the moment they touch. The sensor outputs spatially aligned visual-tactile images. Vision-language models can read them directly. Here's the paper: Changyi Lin and team - hats off ;) ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:47

Seeing touch without pressing down! 🫆 Most tactile sensors rely on deformation. Press down, deform the sensor, measure the deformation. But light contact, touching water, cream, soft films, doesn't deform anything. The sensor sees nothing. LightTact flips the approach. It uses optical detection instead. An ambient-blocking configuration suppresses external light and internal illumination everywhere except at actual contact points. Only scattered light from true contact gets through. The results? High-contrast raw images. Non-contact pixels stay near-black. Contact pixels show the natural appearance of whatever's touching. This works across material properties, contact forces, surface colors, and lighting conditions. Water, facial cream, thin films, liquids, all become visible the moment they touch. The sensor outputs spatially aligned visual-tactile images. Vision-language models can read them directly. Here's the paper: Changyi Lin and team - hats off ;) ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

207,951 次观看 • 4 个月前

Manipulation happens through surfaces. To understand contact-rich dexterous interaction, motion alone is not enough. We also need to know surface properties and contact state. Excited to share ART-Glove, an articulated tactile glove that captures contact-grounded information while preserving human dexterity. It provides: - Known Geometry: 16 rigid functional surfaces - Surface Motion: 22 anatomically aligned joints - Tactile Contact: 2048 piezoresistive taxels Huge thanks to my advisor Ding Ding Zhao, and to Yuxiang Yuxiang Yang, Maria Maria Bauza Villalonga, Marissa, and Peide Peide Huang for the valuable advice and discussions. Paper:
2:24

Manipulation happens through surfaces. To understand contact-rich dexterous interaction, motion alone is not enough. We also need to know surface properties and contact state. Excited to share ART-Glove, an articulated tactile glove that captures contact-grounded information while preserving human dexterity. It provides: - Known Geometry: 16 rigid functional surfaces - Surface Motion: 22 anatomically aligned joints - Tactile Contact: 2048 piezoresistive taxels Huge thanks to my advisor Ding Ding Zhao, and to Yuxiang Yuxiang Yang, Maria Maria Bauza Villalonga, Marissa, and Peide Peide Huang for the valuable advice and discussions. Paper:

Changyi Lin

90,269 次观看 • 5 个月前

In house tactile sensor prototype. Detects taps, motion, and multiple contact with force magnitudes. Built to be cheap, robust, and replaceable. Not in our demos yet, still WIP. We pair simple tactile sensing with torque transparent joints to recover rich contact without expensive sensors.
0:36

In house tactile sensor prototype. Detects taps, motion, and multiple contact with force magnitudes. Built to be cheap, robust, and replaceable. Not in our demos yet, still WIP. We pair simple tactile sensing with torque transparent joints to recover rich contact without expensive sensors.

Kyber Labs

39,735 次观看 • 7 个月前

Tactile sim-to-real has been the cost floor for a decade. Cheap to simulate a world, impossible to simulate a fingertip. So the field routed around it with grippers and vision only: good demos, no generalization. Sharpa’s TacMap changes the math: map object-to-sensor penetration depth to a 2D deformation image. Contact error 85%. Tactile data stops being a collection problem and becomes a compute problem. And compute scales. The teams that win dexterity won't log the most teleoperated hours. They'll generate contact data faster than rivals can collect it. h/t:
0:20

Tactile sim-to-real has been the cost floor for a decade. Cheap to simulate a world, impossible to simulate a fingertip. So the field routed around it with grippers and vision only: good demos, no generalization. Sharpa’s TacMap changes the math: map object-to-sensor penetration depth to a 2D deformation image. Contact error 85%. Tactile data stops being a collection problem and becomes a compute problem. And compute scales. The teams that win dexterity won't log the most teleoperated hours. They'll generate contact data faster than rivals can collect it. h/t:

Cybernetic Labs

15,305 次观看 • 3 个月前

Introducing LocoTouch: Quadrupedal robots equipped with tactile sensing can now transport unsecured objects — no mounts, no straps. The tactile policy transfers zero-shot from sim to real. Core Task-Agnostic Features: 1. High-fidelity contact simulation for distributed tactile sensors 2. Adaptive gait reward for symmetric, frequency-adaptive locomotion Website: Paper:
1:50

Introducing LocoTouch: Quadrupedal robots equipped with tactile sensing can now transport unsecured objects — no mounts, no straps. The tactile policy transfers zero-shot from sim to real. Core Task-Agnostic Features: 1. High-fidelity contact simulation for distributed tactile sensors 2. Adaptive gait reward for symmetric, frequency-adaptive locomotion Website: Paper:

Changyi Lin

38,782 次观看 • 1 年前

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.
2:35

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.

AI at Meta

453,988 次观看 • 2 年前

For a player to be ruled down by contact, the defender’s contact does not have to be the reason the player goes to the ground. If contact occurs and the player subsequently goes down, the player is still ruled down by contact.
0:27

For a player to be ruled down by contact, the defender’s contact does not have to be the reason the player goes to the ground. If contact occurs and the player subsequently goes down, the player is still ruled down by contact.

Gene Steratore

516,640 次观看 • 2 个月前

Robot hands still break most simulators. Meta Reality Labs just open-sourced Project SuperDex — a contact-first physics engine built for the hard part: multi-finger grasps, in-hand reorientation, soft contact, tactile sensing. VR teleop for cheap demo data. Gym-style RL on top. projectsuperdex. com
0:20

Robot hands still break most simulators. Meta Reality Labs just open-sourced Project SuperDex — a contact-first physics engine built for the hard part: multi-finger grasps, in-hand reorientation, soft contact, tactile sensing. VR teleop for cheap demo data. Gym-style RL on top. projectsuperdex. com

Carlos E. Perez

27,810 次观看 • 2 个月前

Tactile Diffusion generates synthetic tactile images from sim data, capturing the complex illumination of the gel deformation. This research from UW & Meta AI is the first method using diffusion to close the sim2real gap for vision-based tactile sensing. Read the paper ⬇️
1:23

Tactile Diffusion generates synthetic tactile images from sim data, capturing the complex illumination of the gel deformation. This research from UW & Meta AI is the first method using diffusion to close the sim2real gap for vision-based tactile sensing. Read the paper ⬇️

AI at Meta

100,168 次观看 • 3 年前

Presenting research from Berkeley AI Research Humanoid Intelligence Center. DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation.🖐️🤖 Dexterous manipulation requires touch, yet multi-finger tactile data remain scarce and expensive to collect at scale. Through continual vision-to-touch learning, DexTacWAM adapts a pretrained video world model into a visuo-tactile world model for predictive multi-finger contact modeling and action generation. Across six contact-rich dexterous manipulation tasks, DexTacWAM achieves the highest score on every task, averaging 70.6 vs. 38.0 for the strongest baseline. 🔬 Why not simply inject tactile features into the action policy? With the same multi-finger tactile encoder, policy architecture, and training procedure, replacing the tactile world-model latent with direct tactile features drops the 4-task mean from 74.7 to 26.6. Takeaway: the benefit does not come simply from providing tactile observations to the policy, but from making multi-finger contact evolution part of the predicted world state. All code, model weights, and datasets are now open-sourced! 🌐 Project: 📄 Paper: 💻 Code: 🤗 Weights&Data: Author List: Haoran Yuan✈️Corl, Zekai Wang, Boning Shao Haoran Lu, trevordarrell, Ismini Lourentzou, Wei ZHAN
2:24

Presenting research from Berkeley AI Research Humanoid Intelligence Center. DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation.🖐️🤖 Dexterous manipulation requires touch, yet multi-finger tactile data remain scarce and expensive to collect at scale. Through continual vision-to-touch learning, DexTacWAM adapts a pretrained video world model into a visuo-tactile world model for predictive multi-finger contact modeling and action generation. Across six contact-rich dexterous manipulation tasks, DexTacWAM achieves the highest score on every task, averaging 70.6 vs. 38.0 for the strongest baseline. 🔬 Why not simply inject tactile features into the action policy? With the same multi-finger tactile encoder, policy architecture, and training procedure, replacing the tactile world-model latent with direct tactile features drops the 4-task mean from 74.7 to 26.6. Takeaway: the benefit does not come simply from providing tactile observations to the policy, but from making multi-finger contact evolution part of the predicted world state. All code, model weights, and datasets are now open-sourced! 🌐 Project: 📄 Paper: 💻 Code: 🤗 Weights&Data: Author List: Haoran Yuan✈️Corl, Zekai Wang, Boning Shao Haoran Lu, trevordarrell, Ismini Lourentzou, Wei ZHAN

Berkeley AI Research

115,460 次观看 • 1 个月前

Sharpa Robotics just dropped a new hand video and the level keeps going up. This is the Sharpa Wave running WM Craftnet on a human scale fivefinger hand with 22 active DoF. The policy combines wrist depth, tactile sensing, proprioception and previous actions. The hand can rotate different objects in-hand, recover after external pushes and continue manipulating objects it was never trained on. The numbers are strong. 175/200 successful real-world rotation trials across 20 objects. A world-model prior trained on 9 objects was transferred to 49 new objects. Fall rate went from 6% to 0.3%. The Wave hardware itself has 22 actuators, up to 20 N fingertip force, 240×240 tactile sensing at up to 180 fps and 0.02 N pressure sensitivity. What caught my attention is the recovery behavior. The fingers keep changing contact points after the object slips or gets pushed instead of replaying the same finger motion. That is the kind of dexterity I want to see more of in robotic hands. According to Sharpa’s current specifications: • DTA tactile sensors on the fingers with a resolution of up to 240 × 240 • Pressure detection • Slip detection • Force change detection • Contact point localization • 6-axis force and torque measurement: Fx, Fy, Fz, Mx, My, Mz • Tactile sensing at up to 180 fps • 20 ms reported latency • Force detection range from 0 to 30 N • Maximum sensor load of 50 N • Sharpa also describes a miniature camera integrated into each fingertip for visuo-tactile sensing.
0:33

Sharpa Robotics just dropped a new hand video and the level keeps going up. This is the Sharpa Wave running WM Craftnet on a human scale fivefinger hand with 22 active DoF. The policy combines wrist depth, tactile sensing, proprioception and previous actions. The hand can rotate different objects in-hand, recover after external pushes and continue manipulating objects it was never trained on. The numbers are strong. 175/200 successful real-world rotation trials across 20 objects. A world-model prior trained on 9 objects was transferred to 49 new objects. Fall rate went from 6% to 0.3%. The Wave hardware itself has 22 actuators, up to 20 N fingertip force, 240×240 tactile sensing at up to 180 fps and 0.02 N pressure sensitivity. What caught my attention is the recovery behavior. The fingers keep changing contact points after the object slips or gets pushed instead of replaying the same finger motion. That is the kind of dexterity I want to see more of in robotic hands. According to Sharpa’s current specifications: • DTA tactile sensors on the fingers with a resolution of up to 240 × 240 • Pressure detection • Slip detection • Force change detection • Contact point localization • 6-axis force and torque measurement: Fx, Fy, Fz, Mx, My, Mz • Tactile sensing at up to 180 fps • 20 ms reported latency • Force detection range from 0 to 30 N • Maximum sensor load of 50 N • Sharpa also describes a miniature camera integrated into each fingertip for visuo-tactile sensing.

Techniahqrobot | humanoid robots

13,730 次观看 • 1 个月前

People are stranded out there with no way to contact emergency services, doctors, any other kind of help, or loved ones. Thanks to Elon Musk, Mike Coryell from Starlink and the Samaritan's Purse team, we were able to put a lot of people back in contact with the world. Stay tuned for more updates. A special thanks to Michaels Stores for their contributions to the mission.
1:23

People are stranded out there with no way to contact emergency services, doctors, any other kind of help, or loved ones. Thanks to Elon Musk, Mike Coryell from Starlink and the Samaritan's Purse team, we were able to put a lot of people back in contact with the world. Stay tuned for more updates. A special thanks to Michaels Stores for their contributions to the mission.

Dr. Phil

922,741 次观看 • 2 年前

We have plenty of human demonstration data for movement. We don’t have enough for touch. Without tactile feedback, foundation models can’t feel if an object is slipping, soft, or fragile during complex manipulation. Matrix Inno is bringing three data-collection gloves to IROS 2026: the RigidCore Tactile+EMF, the Fabric Tactile + EMF, and the Fabric Tactile + UltraLite Motion Glove. The hardware tracks pressure and hand pose simultaneously during human demos, mapping exactly where contact happens and how it shifts in real time. Wearables like this are how we finally get real, continuous contact data at scale. Source: Matrix Inno MX MATRIX
0:26

We have plenty of human demonstration data for movement. We don’t have enough for touch. Without tactile feedback, foundation models can’t feel if an object is slipping, soft, or fragile during complex manipulation. Matrix Inno is bringing three data-collection gloves to IROS 2026: the RigidCore Tactile+EMF, the Fabric Tactile + EMF, and the Fabric Tactile + UltraLite Motion Glove. The hardware tracks pressure and hand pose simultaneously during human demos, mapping exactly where contact happens and how it shifts in real time. Wearables like this are how we finally get real, continuous contact data at scale. Source: Matrix Inno MX MATRIX

RoboHub🤖

12,372 次观看 • 1 个月前

For those who think the ball can't be in contact with the palm when getting to set point: I present Simone Fontecchio, who's fingers are not in contact with ball until the moment of the release. Dame and Wemby also do this, to make a few.
0:29

For those who think the ball can't be in contact with the palm when getting to set point: I present Simone Fontecchio, who's fingers are not in contact with ball until the moment of the release. Dame and Wemby also do this, to make a few.

BBALLBREAKDOWN

246,486 次观看 • 1 年前

LB 1-2 Step Tackling Progression - Linebackers should focus on using the near foot and near shoulder when initiating contact with a ball carrier. - Players must overemphasize swinging their arms in an uppercut motion to generate power through the contact point. - A core fundamental is to get a leg and take a leg, which involves squeezing and attempting to lift the opponent's leg during the tackle. - Coaches should look for players to have their face in contact and aimed at the center of the ball carrier's mass. - After making contact, defenders should overemphasize driving their feet for at least five steps to finish the play. - The tackling progression moves systematically from static contact to taking one step into contact and then two steps back to build consistency. AJ Cooper, LB Coach, Arizona State AJ Cooper #glazierclinics
1:19

LB 1-2 Step Tackling Progression - Linebackers should focus on using the near foot and near shoulder when initiating contact with a ball carrier. - Players must overemphasize swinging their arms in an uppercut motion to generate power through the contact point. - A core fundamental is to get a leg and take a leg, which involves squeezing and attempting to lift the opponent's leg during the tackle. - Coaches should look for players to have their face in contact and aimed at the center of the ball carrier's mass. - After making contact, defenders should overemphasize driving their feet for at least five steps to finish the play. - The tackling progression moves systematically from static contact to taking one step into contact and then two steps back to build consistency. AJ Cooper, LB Coach, Arizona State AJ Cooper #glazierclinics

Glazier Clinics

15,300 次观看 • 4 个月前

Open-source magnetic tactile sensor for $5! 🧲 Researchers introduced a magnetic tactile sensor that's low-cost, and easy to fabricate, democratizing tactile sensing for robotics. Operating in unstructured environments like homes and offices requires robots to sense forces during physical interaction. Yet the lack of a versatile, accessible tactile sensor has led to fragmented solutions and often force-unaware, sensorless approaches. Building an eFlesh sensor requires four components: a hobbyist 3D printer, off-the-shelf magnets (less than $5), a CAD model, and a magnetometer circuit board. The sensor is 3D printed with magnets embedded in the middle layer. Based on chosen mechanical properties, magnets displace in response to contact forces, measured by a magnetometer underneath. An open-source design tool converts simple OBJ/STL files into 3D-printable STLs. This enables application-specific sensors for robot hands, grippers, quadruped feet, and more. Slip detection generalizes to unseen objects with 95% accuracy. Visual-tactile control policies improve manipulation by 40% over vision-only baselines, achieving 90% success on precise tasks like plug insertion and credit card swiping. All design files, code, trained models, and conversion tools are openly available. Project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:57

Open-source magnetic tactile sensor for $5! 🧲 Researchers introduced a magnetic tactile sensor that's low-cost, and easy to fabricate, democratizing tactile sensing for robotics. Operating in unstructured environments like homes and offices requires robots to sense forces during physical interaction. Yet the lack of a versatile, accessible tactile sensor has led to fragmented solutions and often force-unaware, sensorless approaches. Building an eFlesh sensor requires four components: a hobbyist 3D printer, off-the-shelf magnets (less than $5), a CAD model, and a magnetometer circuit board. The sensor is 3D printed with magnets embedded in the middle layer. Based on chosen mechanical properties, magnets displace in response to contact forces, measured by a magnetometer underneath. An open-source design tool converts simple OBJ/STL files into 3D-printable STLs. This enables application-specific sensors for robot hands, grippers, quadruped feet, and more. Slip detection generalizes to unseen objects with 95% accuracy. Visual-tactile control policies improve manipulation by 40% over vision-only baselines, achieving 90% success on precise tasks like plug insertion and credit card swiping. All design files, code, trained models, and conversion tools are openly available. Project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

290,388 次观看 • 6 个月前

Holy…S😳 This isn’t fabric. It’s electronic skin,woven with dense fiber and textile sensors that can detect pressure, touch, deformation, and subtle contact changes in real time. Now imagine humanoid robots covered in it,especially on dexterous hands. humanoids could truly touch and understand the physical world and even humans. (from Shanghai JQ INDUSTRIES )
1:46

Holy…S😳 This isn’t fabric. It’s electronic skin,woven with dense fiber and textile sensors that can detect pressure, touch, deformation, and subtle contact changes in real time. Now imagine humanoid robots covered in it,especially on dexterous hands. humanoids could truly touch and understand the physical world and even humans. (from Shanghai JQ INDUSTRIES )

CyberRobo

174,143 次观看 • 8 个月前