Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

In house tactile sensor prototype. Detects taps, motion, and multiple contact with force magnitudes. Built to be cheap, robust, and replaceable. Not in our demos yet, still WIP. We pair simple tactile sensing with torque transparent joints to recover rich contact without expensive sensors.

KyberLabsRobots's profile picture

Kyber Labs

2,935 subscribers

38,930 Aufrufe • vor 6 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Contact does not always deform surfaces, especially when interacting with liquids and soft objects. However, most existing tactile sensors rely on surface deformation to infer contact. Excited to share LightTact, a visual-tactile fingertip sensor that makes contact directly visible. It provides: - Deformation-Independent Sensing: contact is detected optically, not inferred from surface deformation. - High-Contrast Raw Images: non-contact pixels stay near-black. - Pixel-Level Contact Segmentation: robust across materials, forces, appearances, and external lighting conditions. This project was co-led with Boda ( Grateful to all co-authors for their contributions, and many thanks to Yuxiang Yang for the insightful discussions. We will present LightTact at RSS 2026 in Sydney next week! Paper: Open-source:
0:52

Contact does not always deform surfaces, especially when interacting with liquids and soft objects. However, most existing tactile sensors rely on surface deformation to infer contact. Excited to share LightTact, a visual-tactile fingertip sensor that makes contact directly visible. It provides: - Deformation-Independent Sensing: contact is detected optically, not inferred from surface deformation. - High-Contrast Raw Images: non-contact pixels stay near-black. - Pixel-Level Contact Segmentation: robust across materials, forces, appearances, and external lighting conditions. This project was co-led with Boda ( Grateful to all co-authors for their contributions, and many thanks to Yuxiang Yang for the insightful discussions. We will present LightTact at RSS 2026 in Sydney next week! Paper: Open-source:

Changyi Lin

65,556 Aufrufe • vor 3 Monaten

Manipulation happens through surfaces. To understand contact-rich dexterous interaction, motion alone is not enough. We also need to know surface properties and contact state. Excited to share ART-Glove, an articulated tactile glove that captures contact-grounded information while preserving human dexterity. It provides: - Known Geometry: 16 rigid functional surfaces - Surface Motion: 22 anatomically aligned joints - Tactile Contact: 2048 piezoresistive taxels Huge thanks to my advisor Ding Ding Zhao, and to Yuxiang Yuxiang Yang, Maria Maria Bauza Villalonga, Marissa, and Peide Peide Huang for the valuable advice and discussions. Paper:
2:24

Manipulation happens through surfaces. To understand contact-rich dexterous interaction, motion alone is not enough. We also need to know surface properties and contact state. Excited to share ART-Glove, an articulated tactile glove that captures contact-grounded information while preserving human dexterity. It provides: - Known Geometry: 16 rigid functional surfaces - Surface Motion: 22 anatomically aligned joints - Tactile Contact: 2048 piezoresistive taxels Huge thanks to my advisor Ding Ding Zhao, and to Yuxiang Yuxiang Yang, Maria Maria Bauza Villalonga, Marissa, and Peide Peide Huang for the valuable advice and discussions. Paper:

Changyi Lin

88,658 Aufrufe • vor 3 Monaten

ManiFeel is a benchmark for contact-rich manipulation tasks. Force- and tactile-based tasks with occlusions and partially-observable state are going to be crucial for true general purpose robotics. Website:
0:20

ManiFeel is a benchmark for contact-rich manipulation tasks. Force- and tactile-based tasks with occlusions and partially-observable state are going to be crucial for true general purpose robotics. Website:

Chris Paxton

12,063 Aufrufe • vor 7 Monaten

Now all you need to make tactile sensors is a 3D printer, magnets, and magnetometers! [📍It’s open source!] A new tactile sensor, called e-Flesh, with a simple working principle… measure deformations in 3D printable microstructures. Since e-Flesh is 3D printable, you can make it in all shapes and sizes for applications ranging from foot fall sensing to multifingered hands. This is critical in getting touch not just on fingertips, but all around robots. eFlesh can democratize touch sensing with open-sourced❗️ Make your own: Paper:
0:57

Now all you need to make tactile sensors is a 3D printer, magnets, and magnetometers! [📍It’s open source!] A new tactile sensor, called e-Flesh, with a simple working principle… measure deformations in 3D printable microstructures. Since e-Flesh is 3D printable, you can make it in all shapes and sizes for applications ranging from foot fall sensing to multifingered hands. This is critical in getting touch not just on fingertips, but all around robots. eFlesh can democratize touch sensing with open-sourced❗️ Make your own: Paper:

Ilir Aliu

154,953 Aufrufe • vor 8 Monaten

Sharpa Robotics just dropped a new hand video and the level keeps going up. This is the Sharpa Wave running WM Craftnet on a human scale fivefinger hand with 22 active DoF. The policy combines wrist depth, tactile sensing, proprioception and previous actions. The hand can rotate different objects in-hand, recover after external pushes and continue manipulating objects it was never trained on. The numbers are strong. 175/200 successful real-world rotation trials across 20 objects. A world-model prior trained on 9 objects was transferred to 49 new objects. Fall rate went from 6% to 0.3%. The Wave hardware itself has 22 actuators, up to 20 N fingertip force, 240×240 tactile sensing at up to 180 fps and 0.02 N pressure sensitivity. What caught my attention is the recovery behavior. The fingers keep changing contact points after the object slips or gets pushed instead of replaying the same finger motion. That is the kind of dexterity I want to see more of in robotic hands. According to Sharpa’s current specifications: • DTA tactile sensors on the fingers with a resolution of up to 240 × 240 • Pressure detection • Slip detection • Force change detection • Contact point localization • 6-axis force and torque measurement: Fx, Fy, Fz, Mx, My, Mz • Tactile sensing at up to 180 fps • 20 ms reported latency • Force detection range from 0 to 30 N • Maximum sensor load of 50 N • Sharpa also describes a miniature camera integrated into each fingertip for visuo-tactile sensing.
0:33

Sharpa Robotics just dropped a new hand video and the level keeps going up. This is the Sharpa Wave running WM Craftnet on a human scale fivefinger hand with 22 active DoF. The policy combines wrist depth, tactile sensing, proprioception and previous actions. The hand can rotate different objects in-hand, recover after external pushes and continue manipulating objects it was never trained on. The numbers are strong. 175/200 successful real-world rotation trials across 20 objects. A world-model prior trained on 9 objects was transferred to 49 new objects. Fall rate went from 6% to 0.3%. The Wave hardware itself has 22 actuators, up to 20 N fingertip force, 240×240 tactile sensing at up to 180 fps and 0.02 N pressure sensitivity. What caught my attention is the recovery behavior. The fingers keep changing contact points after the object slips or gets pushed instead of replaying the same finger motion. That is the kind of dexterity I want to see more of in robotic hands. According to Sharpa’s current specifications: • DTA tactile sensors on the fingers with a resolution of up to 240 × 240 • Pressure detection • Slip detection • Force change detection • Contact point localization • 6-axis force and torque measurement: Fx, Fy, Fz, Mx, My, Mz • Tactile sensing at up to 180 fps • 20 ms reported latency • Force detection range from 0 to 30 N • Maximum sensor load of 50 N • Sharpa also describes a miniature camera integrated into each fingertip for visuo-tactile sensing.

Techniahqrobot | humanoid robots

13,500 Aufrufe • vor 17 Tagen

Open-source magnetic tactile sensor for $5! 🧲 Researchers from New York University introduced a magnetic tactile sensor that's low-cost, and easy to fabricate, democratizing tactile sensing for robotics. Operating in unstructured environments like homes and offices requires robots to sense forces during physical interaction. Yet the lack of a versatile, accessible tactile sensor has led to fragmented solutions and often force-unaware, sensorless approaches. Building an eFlesh sensor requires four components: a hobbyist 3D printer, off-the-shelf magnets (less than $5), a CAD model, and a magnetometer circuit board. The sensor is 3D printed with magnets embedded in the middle layer. Based on chosen mechanical properties, magnets displace in response to contact forces, measured by a magnetometer underneath. An open-source design tool converts simple OBJ/STL files into 3D-printable STLs. This enables application-specific sensors for robot hands, grippers, quadruped feet, and more. Slip detection generalizes to unseen objects with 95% accuracy. Visual-tactile control policies improve manipulation by 40% over vision-only baselines, achieving 90% success on precise tasks like plug insertion and credit card swiping. All design files, code, trained models, and conversion tools are openly available. 🔗 Project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:57

Open-source magnetic tactile sensor for $5! 🧲 Researchers from New York University introduced a magnetic tactile sensor that's low-cost, and easy to fabricate, democratizing tactile sensing for robotics. Operating in unstructured environments like homes and offices requires robots to sense forces during physical interaction. Yet the lack of a versatile, accessible tactile sensor has led to fragmented solutions and often force-unaware, sensorless approaches. Building an eFlesh sensor requires four components: a hobbyist 3D printer, off-the-shelf magnets (less than $5), a CAD model, and a magnetometer circuit board. The sensor is 3D printed with magnets embedded in the middle layer. Based on chosen mechanical properties, magnets displace in response to contact forces, measured by a magnetometer underneath. An open-source design tool converts simple OBJ/STL files into 3D-printable STLs. This enables application-specific sensors for robot hands, grippers, quadruped feet, and more. Slip detection generalizes to unseen objects with 95% accuracy. Visual-tactile control policies improve manipulation by 40% over vision-only baselines, achieving 90% success on precise tasks like plug insertion and credit card swiping. All design files, code, trained models, and conversion tools are openly available. 🔗 Project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

117,301 Aufrufe • vor 2 Tagen

Open-source magnetic tactile sensor for $5! 🧲 Researchers introduced a magnetic tactile sensor that's low-cost, and easy to fabricate, democratizing tactile sensing for robotics. Operating in unstructured environments like homes and offices requires robots to sense forces during physical interaction. Yet the lack of a versatile, accessible tactile sensor has led to fragmented solutions and often force-unaware, sensorless approaches. Building an eFlesh sensor requires four components: a hobbyist 3D printer, off-the-shelf magnets (less than $5), a CAD model, and a magnetometer circuit board. The sensor is 3D printed with magnets embedded in the middle layer. Based on chosen mechanical properties, magnets displace in response to contact forces, measured by a magnetometer underneath. An open-source design tool converts simple OBJ/STL files into 3D-printable STLs. This enables application-specific sensors for robot hands, grippers, quadruped feet, and more. Slip detection generalizes to unseen objects with 95% accuracy. Visual-tactile control policies improve manipulation by 40% over vision-only baselines, achieving 90% success on precise tasks like plug insertion and credit card swiping. All design files, code, trained models, and conversion tools are openly available. Project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:57

Open-source magnetic tactile sensor for $5! 🧲 Researchers introduced a magnetic tactile sensor that's low-cost, and easy to fabricate, democratizing tactile sensing for robotics. Operating in unstructured environments like homes and offices requires robots to sense forces during physical interaction. Yet the lack of a versatile, accessible tactile sensor has led to fragmented solutions and often force-unaware, sensorless approaches. Building an eFlesh sensor requires four components: a hobbyist 3D printer, off-the-shelf magnets (less than $5), a CAD model, and a magnetometer circuit board. The sensor is 3D printed with magnets embedded in the middle layer. Based on chosen mechanical properties, magnets displace in response to contact forces, measured by a magnetometer underneath. An open-source design tool converts simple OBJ/STL files into 3D-printable STLs. This enables application-specific sensors for robot hands, grippers, quadruped feet, and more. Slip detection generalizes to unseen objects with 95% accuracy. Visual-tactile control policies improve manipulation by 40% over vision-only baselines, achieving 90% success on precise tasks like plug insertion and credit card swiping. All design files, code, trained models, and conversion tools are openly available. Project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

289,657 Aufrufe • vor 5 Monaten

Tactile sim-to-real has been the cost floor for a decade. Cheap to simulate a world, impossible to simulate a fingertip. So the field routed around it with grippers and vision only: good demos, no generalization. Sharpa’s TacMap changes the math: map object-to-sensor penetration depth to a 2D deformation image. Contact error 85%. Tactile data stops being a collection problem and becomes a compute problem. And compute scales. The teams that win dexterity won't log the most teleoperated hours. They'll generate contact data faster than rivals can collect it. h/t:
0:20

Tactile sim-to-real has been the cost floor for a decade. Cheap to simulate a world, impossible to simulate a fingertip. So the field routed around it with grippers and vision only: good demos, no generalization. Sharpa’s TacMap changes the math: map object-to-sensor penetration depth to a 2D deformation image. Contact error 85%. Tactile data stops being a collection problem and becomes a compute problem. And compute scales. The teams that win dexterity won't log the most teleoperated hours. They'll generate contact data faster than rivals can collect it. h/t:

Cybernetic Labs

15,305 Aufrufe • vor 2 Monaten

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.
2:35

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.

AI at Meta

453,860 Aufrufe • vor 1 Jahr

Presenting research from Berkeley AI Research Humanoid Intelligence Center. DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation.🖐️🤖 Dexterous manipulation requires touch, yet multi-finger tactile data remain scarce and expensive to collect at scale. Through continual vision-to-touch learning, DexTacWAM adapts a pretrained video world model into a visuo-tactile world model for predictive multi-finger contact modeling and action generation. Across six contact-rich dexterous manipulation tasks, DexTacWAM achieves the highest score on every task, averaging 70.6 vs. 38.0 for the strongest baseline. 🔬 Why not simply inject tactile features into the action policy? With the same multi-finger tactile encoder, policy architecture, and training procedure, replacing the tactile world-model latent with direct tactile features drops the 4-task mean from 74.7 to 26.6. Takeaway: the benefit does not come simply from providing tactile observations to the policy, but from making multi-finger contact evolution part of the predicted world state. All code, model weights, and datasets are now open-sourced! 🌐 Project: 📄 Paper: 💻 Code: 🤗 Weights&Data: Author List: Haoran Yuan, Zekai Wang, Boning Shao Haoran Lu, trevordarrell, Ismini Lourentzou, Wei ZHAN
2:24

Presenting research from Berkeley AI Research Humanoid Intelligence Center. DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation.🖐️🤖 Dexterous manipulation requires touch, yet multi-finger tactile data remain scarce and expensive to collect at scale. Through continual vision-to-touch learning, DexTacWAM adapts a pretrained video world model into a visuo-tactile world model for predictive multi-finger contact modeling and action generation. Across six contact-rich dexterous manipulation tasks, DexTacWAM achieves the highest score on every task, averaging 70.6 vs. 38.0 for the strongest baseline. 🔬 Why not simply inject tactile features into the action policy? With the same multi-finger tactile encoder, policy architecture, and training procedure, replacing the tactile world-model latent with direct tactile features drops the 4-task mean from 74.7 to 26.6. Takeaway: the benefit does not come simply from providing tactile observations to the policy, but from making multi-finger contact evolution part of the predicted world state. All code, model weights, and datasets are now open-sourced! 🌐 Project: 📄 Paper: 💻 Code: 🤗 Weights&Data: Author List: Haoran Yuan, Zekai Wang, Boning Shao Haoran Lu, trevordarrell, Ismini Lourentzou, Wei ZHAN

Berkeley AI Research

109,869 Aufrufe • vor 7 Tagen

Excited to share that our work NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception has been accepted to #RSS2026! Your robot — even a low-cost one — can feel external forces without torque or tactile sensors. TL;DR: NeuralActuator is a neural actuator model that jointly predicts 1️⃣torque to capture the nonlinear and time-varying current–to–torque relationship of low-cost servos, 2️⃣external contact forces (and force detection gates) for sensorless force perception, 3️⃣and motor conditions that indicate each motor’s operating regime. Here is a fast-forward video clip ⬇️ We are also covering more robots like LeRobot-S101 and Franka Panda. More details coming soon.
1:46

Excited to share that our work NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception has been accepted to #RSS2026! Your robot — even a low-cost one — can feel external forces without torque or tactile sensors. TL;DR: NeuralActuator is a neural actuator model that jointly predicts 1️⃣torque to capture the nonlinear and time-varying current–to–torque relationship of low-cost servos, 2️⃣external contact forces (and force detection gates) for sensorless force perception, 3️⃣and motor conditions that indicate each motor’s operating regime. Here is a fast-forward video clip ⬇️ We are also covering more robots like LeRobot-S101 and Franka Panda. More details coming soon.

Zhiyang (Frank) Dou

51,006 Aufrufe • vor 5 Monaten

Who can say no to a humanoid robot that can actually feel a hug? 🤖 JQ Industries just built a fabric-based electronic skin for Unitree G1 — covering the torso, hands, and feet. This is not about appearance. It’s about giving robots real tactile sensing and pressure feedback. According to the company, the system includes tactile gloves, upper-body skin, and foot sensors. That means the robot can feel contact through its hands, detect pressure across its body, and use foot feedback to adjust balance. This is the missing layer in most humanoids. Vision alone can’t handle physical interaction. Robots need touch to grip safely, react to humans, and stay stable in real-world environments. JQ says its fiber sensors can detect forces as low as 0.01N, handle up to 3500kPa, and bend below 0.1mm radius. The system can also integrate with Isaac Sim, making tactile data usable for training and deployment. Unitree G1 is already built for manipulation and control — this adds the sensing layer it lacked. JQ says it’s now working with multiple humanoid companies, with some systems already shipping at scale. This is what happens when robots stop just seeing the world — and start physically feeling it.
1:56

Who can say no to a humanoid robot that can actually feel a hug? 🤖 JQ Industries just built a fabric-based electronic skin for Unitree G1 — covering the torso, hands, and feet. This is not about appearance. It’s about giving robots real tactile sensing and pressure feedback. According to the company, the system includes tactile gloves, upper-body skin, and foot sensors. That means the robot can feel contact through its hands, detect pressure across its body, and use foot feedback to adjust balance. This is the missing layer in most humanoids. Vision alone can’t handle physical interaction. Robots need touch to grip safely, react to humans, and stay stable in real-world environments. JQ says its fiber sensors can detect forces as low as 0.01N, handle up to 3500kPa, and bend below 0.1mm radius. The system can also integrate with Isaac Sim, making tactile data usable for training and deployment. Unitree G1 is already built for manipulation and control — this adds the sensing layer it lacked. JQ says it’s now working with multiple humanoid companies, with some systems already shipping at scale. This is what happens when robots stop just seeing the world — and start physically feeling it.

RoboHub🤖

10,952 Aufrufe • vor 6 Monaten

Can we bring human-like Touch to robots🤖? Introducing our CoRL work on 3D-ViTac. Humans rely on both vision 👁️ and touch 🫳 for complex tasks. With combined visual-tactile sensing, robots can now tackle challenging tasks, like precise in-hand reorientation, fragile objects grasping. Website: #Robotics #CoRL2024 #Touch #tactile #AI #ML
0:22

Can we bring human-like Touch to robots🤖? Introducing our CoRL work on 3D-ViTac. Humans rely on both vision 👁️ and touch 🫳 for complex tasks. With combined visual-tactile sensing, robots can now tackle challenging tasks, like precise in-hand reorientation, fragile objects grasping. Website: #Robotics #CoRL2024 #Touch #tactile #AI #ML

Binghao Huang

49,714 Aufrufe • vor 1 Jahr

The sense of touch is the most criminally under-explored modality in robotics. Imagine doing sleight of hand wearing thick oven mitts. That's exactly how a robot feels today if it were alive. A magnetic piece snapping into place, a paper cup peeling out of a stack, a USB negotiating its way into the port - all invisible to the camera. Learning how to feel must be a full-stack co-designed effort. We are open-sourcing a principled methodology called "T-Rex": 1. Tactile as first-class citizen of the model. Our mixture-of-transformer runs two clocks asynchronously: a slow visuomotor expert plans the motion, and a fast tactile expert refines it in real time with high-frequency corrections at 4 "touch ticks" per vision tick. Forces change faster than frames arrive, so the architecture had to as well. 2. Open data. The largest tactile dataset ever released to our knowledge: a 50-hour (~5,500 episodes) high-quality, carefully synchronized robot play corpus, collected on SOTA tactile hand hardware with 22 degrees of freedom. Available today on HuggingFace! 3. Training recipe: T-Rex extends our prior work, EgoScale. Human egocentric videos for pretraining, a diverse dose of tactile robot play for mid-training. Our experiments show this bridges contact-free pretraining to contact-rich manipulation remarkably well. Pixels are cheap and everywhere, but they run out of steam at the moment of contact. Tactile will carry the last mile. The next scaling curve will be measured in hours of touch. T-Rex is a great collaboration between NVIDIA and Berkeley: 🧵
2:21

The sense of touch is the most criminally under-explored modality in robotics. Imagine doing sleight of hand wearing thick oven mitts. That's exactly how a robot feels today if it were alive. A magnetic piece snapping into place, a paper cup peeling out of a stack, a USB negotiating its way into the port - all invisible to the camera. Learning how to feel must be a full-stack co-designed effort. We are open-sourcing a principled methodology called "T-Rex": 1. Tactile as first-class citizen of the model. Our mixture-of-transformer runs two clocks asynchronously: a slow visuomotor expert plans the motion, and a fast tactile expert refines it in real time with high-frequency corrections at 4 "touch ticks" per vision tick. Forces change faster than frames arrive, so the architecture had to as well. 2. Open data. The largest tactile dataset ever released to our knowledge: a 50-hour (~5,500 episodes) high-quality, carefully synchronized robot play corpus, collected on SOTA tactile hand hardware with 22 degrees of freedom. Available today on HuggingFace! 3. Training recipe: T-Rex extends our prior work, EgoScale. Human egocentric videos for pretraining, a diverse dose of tactile robot play for mid-training. Our experiments show this bridges contact-free pretraining to contact-rich manipulation remarkably well. Pixels are cheap and everywhere, but they run out of steam at the moment of contact. Tactile will carry the last mile. The next scaling curve will be measured in hours of touch. T-Rex is a great collaboration between NVIDIA and Berkeley: 🧵

Jim Fan

179,899 Aufrufe • vor 1 Monat

I was really impressed by the UMI gripper (Cheng Chi et al.), but a key limitation is that **force-related data wasn’t captured**: humans feel haptic feedback through the mechanical springs, but the robot couldn’t leverage that info, limiting the data’s value for fine-grained manipulation tasks. Led by my amazing students Yolanda Zhu and Binghao Huang, we designed a **portable visuo-tactile gripper** by integrating our dense, flexible tactile arrays with the UMI gripper to enable large-scale in-the-wild data collection. 🔗 We demonstrate **cross-modal representation learning** and **downstream policy learning** on tasks requiring in-hand state estimation (e.g., test tube reorientation) and fine-grained force sensing (e.g., pipette fluid transfer). Key takeaways: - Our flexible tactile arrays store the rich haptic information humans perceive as dense tactile signals. - Portability and robustness are key for in-the-wild data collection; our portable gripper is compact, lightweight, and durable. - Touch provides precise, robust measurements of in-hand object pose, invariant to lighting and viewpoint. - Cross-modal pretraining on large-scale in-the-wild data significantly improves policy robustness and sample efficiency (as shown many times before — and verified again here!). Also check out our previous investigations of dense, flexible tactile grids for understanding human-robot-environment interactions: - Dense tactile glove (Nature ’19): - 3D-ViTac (CoRL ’24):
5:35

I was really impressed by the UMI gripper (Cheng Chi et al.), but a key limitation is that **force-related data wasn’t captured**: humans feel haptic feedback through the mechanical springs, but the robot couldn’t leverage that info, limiting the data’s value for fine-grained manipulation tasks. Led by my amazing students Yolanda Zhu and Binghao Huang, we designed a **portable visuo-tactile gripper** by integrating our dense, flexible tactile arrays with the UMI gripper to enable large-scale in-the-wild data collection. 🔗 We demonstrate **cross-modal representation learning** and **downstream policy learning** on tasks requiring in-hand state estimation (e.g., test tube reorientation) and fine-grained force sensing (e.g., pipette fluid transfer). Key takeaways: - Our flexible tactile arrays store the rich haptic information humans perceive as dense tactile signals. - Portability and robustness are key for in-the-wild data collection; our portable gripper is compact, lightweight, and durable. - Touch provides precise, robust measurements of in-hand object pose, invariant to lighting and viewpoint. - Cross-modal pretraining on large-scale in-the-wild data significantly improves policy robustness and sample efficiency (as shown many times before — and verified again here!). Also check out our previous investigations of dense, flexible tactile grids for understanding human-robot-environment interactions: - Dense tactile glove (Nature ’19): - 3D-ViTac (CoRL ’24):

Yunzhu Li

13,510 Aufrufe • vor 1 Jahr

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n
1:48

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n

Zhi (Leo) Wang

53,130 Aufrufe • vor 3 Monaten

Force-sensing fingers! 🧤 Stanford researchers just released UMI-FT, a handheld data collection platform that puts compact six-axis force/torque sensors on each finger, enabling finger-level wrench measurements alongside RGB, depth, and pose data. Many manipulation tasks require careful force modulation: too little force and the task fails, too much and you cause damage. But commercial force/torque sensors are expensive, bulky, and fragile, which has limited large-scale force-aware policy learning. UMI-FT changes the economics. The platform uses an iPhone for RGB vision, ultrawide RGB, depth, and pose via ARKit, with each finger sensorized using a CoinFT sensor to capture per-finger wrench information during manipulation. This multimodal data trains an adaptive compliance policy that predicts position targets, grasp force, and stiffness for execution on standard compliance controllers. The learned policy runs slowest and generates reference targets, while model-based compliance and force controllers provide delicate 6D compliance control and real-time force modulation. They tested on three contact-rich, force-sensitive tasks: whiteboard wiping (locate eraser, grasp, wipe until clean), skewering zucchini (grasp slice firmly, push onto stick until punctured), and lightbulb insertion (grasp bulb, align bayonet pin with socket slit, insert while overcoming spring force, rotate to light up). The results are clear. Policies without compliance struggle to modulate contact force and trigger safety faults from excessive force. Policies without force sensing fail to grasp unseen objects or resist reaction forces, causing slippage. Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:29

Force-sensing fingers! 🧤 Stanford researchers just released UMI-FT, a handheld data collection platform that puts compact six-axis force/torque sensors on each finger, enabling finger-level wrench measurements alongside RGB, depth, and pose data. Many manipulation tasks require careful force modulation: too little force and the task fails, too much and you cause damage. But commercial force/torque sensors are expensive, bulky, and fragile, which has limited large-scale force-aware policy learning. UMI-FT changes the economics. The platform uses an iPhone for RGB vision, ultrawide RGB, depth, and pose via ARKit, with each finger sensorized using a CoinFT sensor to capture per-finger wrench information during manipulation. This multimodal data trains an adaptive compliance policy that predicts position targets, grasp force, and stiffness for execution on standard compliance controllers. The learned policy runs slowest and generates reference targets, while model-based compliance and force controllers provide delicate 6D compliance control and real-time force modulation. They tested on three contact-rich, force-sensitive tasks: whiteboard wiping (locate eraser, grasp, wipe until clean), skewering zucchini (grasp slice firmly, push onto stick until punctured), and lightbulb insertion (grasp bulb, align bayonet pin with socket slit, insert while overcoming spring force, rotate to light up). The results are clear. Policies without compliance struggle to modulate contact force and trigger safety faults from excessive force. Policies without force sensing fail to grasp unseen objects or resist reaction forces, causing slippage. Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

12,868 Aufrufe • vor 8 Monaten

TARS has made some big announcements at ICRA 2026. DexHand - A dexterous hand with 1:1 replication of human metacarpal and finger structure, multi-axis thumb opposition - Quasi-direct-drive + backdrivable joints, 21 degrees of freedom - All 26 ASL letter signs reproduced at high precision -0.05mm fingertip tactile sensing - Data. Model. Hand. Fused into one embodied AI-native dexterous hand. OmniVTA with TacForeSight A visuo-tactile world model for general-use, paired with TacForeSight, a force-guided technology that not only allows robots to understand contact but also enables them to predict contact before it happens. - Semantic touch ("slippery," "rough," "soft," "hard") enabling texture-based material cognition. - Operates inside a real-time closed loop - Contact state prediction and reaction at the microsecond scale, faster than human AWE 3.0 - Claimed to be the world's leading general-purpose embodied foundation model for real-world work - A1 robot packing demo as the showcase: long-horizon, bimanual, with autonomous error recovery and end-to-end imitation learning - Set a Guinness World Record in March 2026 for "Most sub-millimeter wire harness assembled by a robot in one hour" - SenseHub data acquisition suite captures human experience data to train AWE 3.0
0:40

TARS has made some big announcements at ICRA 2026. DexHand - A dexterous hand with 1:1 replication of human metacarpal and finger structure, multi-axis thumb opposition - Quasi-direct-drive + backdrivable joints, 21 degrees of freedom - All 26 ASL letter signs reproduced at high precision -0.05mm fingertip tactile sensing - Data. Model. Hand. Fused into one embodied AI-native dexterous hand. OmniVTA with TacForeSight A visuo-tactile world model for general-use, paired with TacForeSight, a force-guided technology that not only allows robots to understand contact but also enables them to predict contact before it happens. - Semantic touch ("slippery," "rough," "soft," "hard") enabling texture-based material cognition. - Operates inside a real-time closed loop - Contact state prediction and reaction at the microsecond scale, faster than human AWE 3.0 - Claimed to be the world's leading general-purpose embodied foundation model for real-world work - A1 robot packing demo as the showcase: long-horizon, bimanual, with autonomous error recovery and end-to-end imitation learning - Set a Guinness World Record in March 2026 for "Most sub-millimeter wire harness assembled by a robot in one hour" - SenseHub data acquisition suite captures human experience data to train AWE 3.0

The Humanoid Hub

11,322 Aufrufe • vor 4 Monaten

Open-source dexterous hands with fingertip sensors! 🪬 ORCA Dexterity just released three dexterous hand models. Their mission: is to democratize robotic hand dexterity. They're sharing progress on exceptional, low-cost hardware and a software layer from low-level control to robotic hand learning. This is the open-source approach to dexterous manipulation. It has 83 taxels per finger with 0.1 N force detection which pretty is impressive for an open-source design. Tactile sensing is critical for dexterous manipulation, knowing contact forces enables gentle grasping, slip detection, and force-controlled assembly. Also 700g weight for the lite version makes it practical for mounting on robot arms without exceeding payload limits. Lower weight means faster movements and lower torque requirements. Open hardware accelerates robotics by letting researchers and builders modify designs for their specific needs without starting from scratch! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:20

Open-source dexterous hands with fingertip sensors! 🪬 ORCA Dexterity just released three dexterous hand models. Their mission: is to democratize robotic hand dexterity. They're sharing progress on exceptional, low-cost hardware and a software layer from low-level control to robotic hand learning. This is the open-source approach to dexterous manipulation. It has 83 taxels per finger with 0.1 N force detection which pretty is impressive for an open-source design. Tactile sensing is critical for dexterous manipulation, knowing contact forces enables gentle grasping, slip detection, and force-controlled assembly. Also 700g weight for the lite version makes it practical for mounting on robot arms without exceeding payload limits. Lower weight means faster movements and lower torque requirements. Open hardware accelerates robotics by letting researchers and builders modify designs for their specific needs without starting from scratch! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

38,306 Aufrufe • vor 6 Monaten

Touch alone isn’t enough. 🖐️ For robotics, tactile intelligence truly levels up when touch gains spatial meaning. At #ICRA2026, we dove into the core concept behind SaTA: Spatially-anchored Tactile Awareness for robust, dexterous manipulation. Read it here: The challenge is fundamental: a robot shouldn’t just register a touch, it needs to understand exactly where that contact occurs relative to its fingers, joints, and overall hand structure. This is the missing link that turns raw data into precise, real-time adjustments during manipulation. Why does this matter? Because the most complex part of any manipulation task happens when vision is at its least reliable, that final millimeter before insertion, sliding, gripping, or fine-tuning. At Sharpa, this is exactly why we’re building tactile hands and tactile AI in tandem. 🚀 #Sharpa #Robotics #EmbodiedAI #TactileIntelligence #ICRA2026 #DexterousManipulation 📷
0:54

Touch alone isn’t enough. 🖐️ For robotics, tactile intelligence truly levels up when touch gains spatial meaning. At #ICRA2026, we dove into the core concept behind SaTA: Spatially-anchored Tactile Awareness for robust, dexterous manipulation. Read it here: The challenge is fundamental: a robot shouldn’t just register a touch, it needs to understand exactly where that contact occurs relative to its fingers, joints, and overall hand structure. This is the missing link that turns raw data into precise, real-time adjustments during manipulation. Why does this matter? Because the most complex part of any manipulation task happens when vision is at its least reliable, that final millimeter before insertion, sliding, gripping, or fine-tuning. At Sharpa, this is exactly why we’re building tactile hands and tactile AI in tandem. 🚀 #Sharpa #Robotics #EmbodiedAI #TactileIntelligence #ICRA2026 #DexterousManipulation 📷

Sharpa

15,402 Aufrufe • vor 4 Monaten