Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Vision alone isn’t enough to solve dexterous manipulation. The sense of touch is needed. UMI-FT uses a custom PCB to provide 6-axis force-torque sensing at each finger for under $10.

simonkalouche's profile picture

Simon Kalouche

10,649 subscribers

20,731 görüntüleme • 7 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

11 Yorum

Simon Kalouche profil fotoğrafı
Simon Kalouche7 ay önce

UMI-FT Project website: CoinFT Project website:

Embedr profil fotoğrafı
Embedr7 ay önce

this could seriously democratize robotics research. no more selling a kidney for decent tactile feedback

MechatronicsEngineer profil fotoğrafı
MechatronicsEngineer7 ay önce

tactile revolution begins

Devesh Vyas profil fotoğrafı
Devesh Vyas7 ay önce

Yeah, robots needs data about torque, force, and other parameters to gauge the situation and perform tasks aptly. Vision alone ain’t enough. Though it could teach a few things, but still data from actuators and other sensors would be the moat going forward

Akshay Kumar profil fotoğrafı
Akshay Kumar7 ay önce

I keep wondering why the biggest labs building VLAs have barely scraped the surface of tactile sensing and joint torque sensing modalities yet! Perhaps a data issue?

Peter Kazanjy profil fotoğrafı
Peter Kazanjy7 ay önce

Sick demo. Going to be used in Nimble?

Sudhir Pratap Yadav profil fotoğrafı
Sudhir Pratap Yadav7 ay önce

Can you explain little bit what is happening in first video. I can its responding to the push but like while maintaining ee position? Like orientation chnage is allowed etc? What kind of controller is it, is it OSC?

Lidoor L. Joseph profil fotoğrafı
Lidoor L. Joseph7 ay önce

This

Gabriele Tinelli profil fotoğrafı
Gabriele Tinelli7 ay önce

Do you think this scales beyond development environments? Will production robots still need expensive FT?

milan 🌞 profil fotoğrafı
milan 🌞7 ay önce

Awesome stuff. I agree, touch is absolutely necessary.

adamyathegreat profil fotoğrafı
adamyathegreat7 ay önce

Cool

Benzer Videolar

Force-sensing fingers! 🧤 Stanford researchers just released UMI-FT, a handheld data collection platform that puts compact six-axis force/torque sensors on each finger, enabling finger-level wrench measurements alongside RGB, depth, and pose data. Many manipulation tasks require careful force modulation: too little force and the task fails, too much and you cause damage. But commercial force/torque sensors are expensive, bulky, and fragile, which has limited large-scale force-aware policy learning. UMI-FT changes the economics. The platform uses an iPhone for RGB vision, ultrawide RGB, depth, and pose via ARKit, with each finger sensorized using a CoinFT sensor to capture per-finger wrench information during manipulation. This multimodal data trains an adaptive compliance policy that predicts position targets, grasp force, and stiffness for execution on standard compliance controllers. The learned policy runs slowest and generates reference targets, while model-based compliance and force controllers provide delicate 6D compliance control and real-time force modulation. They tested on three contact-rich, force-sensitive tasks: whiteboard wiping (locate eraser, grasp, wipe until clean), skewering zucchini (grasp slice firmly, push onto stick until punctured), and lightbulb insertion (grasp bulb, align bayonet pin with socket slit, insert while overcoming spring force, rotate to light up). The results are clear. Policies without compliance struggle to modulate contact force and trigger safety faults from excessive force. Policies without force sensing fail to grasp unseen objects or resist reaction forces, causing slippage. Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:29

Force-sensing fingers! 🧤 Stanford researchers just released UMI-FT, a handheld data collection platform that puts compact six-axis force/torque sensors on each finger, enabling finger-level wrench measurements alongside RGB, depth, and pose data. Many manipulation tasks require careful force modulation: too little force and the task fails, too much and you cause damage. But commercial force/torque sensors are expensive, bulky, and fragile, which has limited large-scale force-aware policy learning. UMI-FT changes the economics. The platform uses an iPhone for RGB vision, ultrawide RGB, depth, and pose via ARKit, with each finger sensorized using a CoinFT sensor to capture per-finger wrench information during manipulation. This multimodal data trains an adaptive compliance policy that predicts position targets, grasp force, and stiffness for execution on standard compliance controllers. The learned policy runs slowest and generates reference targets, while model-based compliance and force controllers provide delicate 6D compliance control and real-time force modulation. They tested on three contact-rich, force-sensitive tasks: whiteboard wiping (locate eraser, grasp, wipe until clean), skewering zucchini (grasp slice firmly, push onto stick until punctured), and lightbulb insertion (grasp bulb, align bayonet pin with socket slit, insert while overcoming spring force, rotate to light up). The results are clear. Policies without compliance struggle to modulate contact force and trigger safety faults from excessive force. Policies without force sensing fail to grasp unseen objects or resist reaction forces, causing slippage. Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

12,822 görüntüleme • 7 ay önce

A modular data glove built for robot learning. ENSURING’s Magic Glove uses quick-release sensor modules, combining 15 motion-tracking units, 1,000+ tactile sensing points, and real-time 6-axis force data. The modular design is the interesting part here: sensors can be swapped or reconfigured depending on the task, making it a more flexible interface for teleoperation, dexterous manipulation, and embodied AI data collection.
1:43

A modular data glove built for robot learning. ENSURING’s Magic Glove uses quick-release sensor modules, combining 15 motion-tracking units, 1,000+ tactile sensing points, and real-time 6-axis force data. The modular design is the interesting part here: sensors can be swapped or reconfigured depending on the task, making it a more flexible interface for teleoperation, dexterous manipulation, and embodied AI data collection.

Eren Chen

33,006 görüntüleme • 29 gün önce

Tactile feedback is one of the most important modalities in manipulation, but has been underutilized in dexterous hands. T-Dex is a framework for learning dexterous policies from tactile play data, beating vision and torque-based methods by 1.7x. 🧵👇
0:55

Tactile feedback is one of the most important modalities in manipulation, but has been underutilized in dexterous hands. T-Dex is a framework for learning dexterous policies from tactile play data, beating vision and torque-based methods by 1.7x. 🧵👇

Lerrel Pinto

84,611 görüntüleme • 3 yıl önce

Open-source dexterous hands with fingertip sensors! 🪬 ORCA Dexterity just released three dexterous hand models. Their mission: is to democratize robotic hand dexterity. They're sharing progress on exceptional, low-cost hardware and a software layer from low-level control to robotic hand learning. This is the open-source approach to dexterous manipulation. It has 83 taxels per finger with 0.1 N force detection which pretty is impressive for an open-source design. Tactile sensing is critical for dexterous manipulation, knowing contact forces enables gentle grasping, slip detection, and force-controlled assembly. Also 700g weight for the lite version makes it practical for mounting on robot arms without exceeding payload limits. Lower weight means faster movements and lower torque requirements. Open hardware accelerates robotics by letting researchers and builders modify designs for their specific needs without starting from scratch! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:20

Open-source dexterous hands with fingertip sensors! 🪬 ORCA Dexterity just released three dexterous hand models. Their mission: is to democratize robotic hand dexterity. They're sharing progress on exceptional, low-cost hardware and a software layer from low-level control to robotic hand learning. This is the open-source approach to dexterous manipulation. It has 83 taxels per finger with 0.1 N force detection which pretty is impressive for an open-source design. Tactile sensing is critical for dexterous manipulation, knowing contact forces enables gentle grasping, slip detection, and force-controlled assembly. Also 700g weight for the lite version makes it practical for mounting on robot arms without exceeding payload limits. Lower weight means faster movements and lower torque requirements. Open hardware accelerates robotics by letting researchers and builders modify designs for their specific needs without starting from scratch! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

38,249 görüntüleme • 6 ay önce

We just released TAVI -- a robotics framework that combines touch and vision to solve challenging dexterous tasks in under 1 hour. The key? Use human demonstrations to initialize a policy, followed by tactile-based online learning with vision-based rewards. Details in🧵(1/7)
0:35

We just released TAVI -- a robotics framework that combines touch and vision to solve challenging dexterous tasks in under 1 hour. The key? Use human demonstrations to initialize a policy, followed by tactile-based online learning with vision-based rewards. Details in🧵(1/7)

Lerrel Pinto

138,931 görüntüleme • 3 yıl önce

Touch alone isn’t enough. 🖐️ For robotics, tactile intelligence truly levels up when touch gains spatial meaning. At #ICRA2026, we dove into the core concept behind SaTA: Spatially-anchored Tactile Awareness for robust, dexterous manipulation. Read it here: The challenge is fundamental: a robot shouldn’t just register a touch, it needs to understand exactly where that contact occurs relative to its fingers, joints, and overall hand structure. This is the missing link that turns raw data into precise, real-time adjustments during manipulation. Why does this matter? Because the most complex part of any manipulation task happens when vision is at its least reliable, that final millimeter before insertion, sliding, gripping, or fine-tuning. At Sharpa, this is exactly why we’re building tactile hands and tactile AI in tandem. 🚀 #Sharpa #Robotics #EmbodiedAI #TactileIntelligence #ICRA2026 #DexterousManipulation 📷
0:54

Touch alone isn’t enough. 🖐️ For robotics, tactile intelligence truly levels up when touch gains spatial meaning. At #ICRA2026, we dove into the core concept behind SaTA: Spatially-anchored Tactile Awareness for robust, dexterous manipulation. Read it here: The challenge is fundamental: a robot shouldn’t just register a touch, it needs to understand exactly where that contact occurs relative to its fingers, joints, and overall hand structure. This is the missing link that turns raw data into precise, real-time adjustments during manipulation. Why does this matter? Because the most complex part of any manipulation task happens when vision is at its least reliable, that final millimeter before insertion, sliding, gripping, or fine-tuning. At Sharpa, this is exactly why we’re building tactile hands and tactile AI in tandem. 🚀 #Sharpa #Robotics #EmbodiedAI #TactileIntelligence #ICRA2026 #DexterousManipulation 📷

Sharpa

15,168 görüntüleme • 3 ay önce

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n
1:48

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n

Zhi (Leo) Wang

52,211 görüntüleme • 2 ay önce

From day one, mimic has been focused on a single goal: general-purpose dexterous manipulation. Today we're proud to announce the mimic hand M1 and the mimic wearable U1. We believe the only way to solve dexterous manipulation at scale is by going full-stack at the frontier of physical AI, building every layer ourselves around one fixed point, the human hand. The M1 is a highly backdrivable, tendon-driven hand that covers the full range of human capability, from heavy payloads to fine manipulation.
1:43

From day one, mimic has been focused on a single goal: general-purpose dexterous manipulation. Today we're proud to announce the mimic hand M1 and the mimic wearable U1. We believe the only way to solve dexterous manipulation at scale is by going full-stack at the frontier of physical AI, building every layer ourselves around one fixed point, the human hand. The M1 is a highly backdrivable, tendon-driven hand that covers the full range of human capability, from heavy payloads to fine manipulation.

mimic

92,695 görüntüleme • 2 ay önce

We went to Wuji's Shenzhen office to get a grip on their dexterous hand design. Each finger joint is driven directly by a tiny motor paired with a worm gear that redirects the axis of rotation. We spoke with a senior team member there to get more insights. Lena RoboHub🤖 Thanks WUJI TECH for the tour!
11:33

We went to Wuji's Shenzhen office to get a grip on their dexterous hand design. Each finger joint is driven directly by a tiny motor paired with a worm gear that redirects the axis of rotation. We spoke with a senior team member there to get more insights. Lena RoboHub🤖 Thanks WUJI TECH for the tour!

The Humanoid Hub

29,546 görüntüleme • 2 ay önce

Jamie Dimon CEO of JP Morgan on raising taxes: “Giving Washington another Trillion isn’t going to solve anything” He uses the word swamp to describe all the special interests. Moderator tried to end his observation. Jaimie is sensing the change of tolerance outside that room for continuing business as usual.
1:20

Jamie Dimon CEO of JP Morgan on raising taxes: “Giving Washington another Trillion isn’t going to solve anything” He uses the word swamp to describe all the special interests. Moderator tried to end his observation. Jaimie is sensing the change of tolerance outside that room for continuing business as usual.

Edward Dowd

85,671 görüntüleme • 7 ay önce

A dexterous hand capable of operating an iPhone Impressive, with nimble fingertip control. INSPIRE ROBOTS from Beijing has released a new dexterous hand featuring tactile and force sensing, and equipped with a motion library, making it ready to use right out of the box. Especially noteworthy is the modular finger design, allowing users to easily replace a damaged finger. With 13 degrees of freedom and a single-hand load capacity of 30kg, it can be used in general applications such as factory component handling, precision assembly, and warehouse logistics sorting. Notably, this company has already sold over 10,000 of these dexterous hands.
1:44

A dexterous hand capable of operating an iPhone Impressive, with nimble fingertip control. INSPIRE ROBOTS from Beijing has released a new dexterous hand featuring tactile and force sensing, and equipped with a motion library, making it ready to use right out of the box. Especially noteworthy is the modular finger design, allowing users to easily replace a damaged finger. With 13 degrees of freedom and a single-hand load capacity of 30kg, it can be used in general applications such as factory component handling, precision assembly, and warehouse logistics sorting. Notably, this company has already sold over 10,000 of these dexterous hands.

CyberRobo

14,196 görüntüleme • 8 ay önce

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:
3:55

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:

Haoran Geng

95,924 görüntüleme • 1 yıl önce

Most of today's AI can see the world, but it doesn’t **feel** it. Capturing the sense of touch is crucial for dexterous robotic manipulation, user modeling, and understanding physical interactions. Introducing OpenTouch: bringing full-hand tactile sensing into real-world AI🖐️ OpenTouch is collected in-the-wild using tactile sensing gloves, hand pose tracking gloves, and egocentric glasses. It includes: • 5 hours of real-world data, • 3 hours densely annotated contact-rich interactions, • 2,900 curated interaction clips, • across 800 objects, 14 environments, and 29 grasp types. all open at:
0:24

Most of today's AI can see the world, but it doesn’t **feel** it. Capturing the sense of touch is crucial for dexterous robotic manipulation, user modeling, and understanding physical interactions. Introducing OpenTouch: bringing full-hand tactile sensing into real-world AI🖐️ OpenTouch is collected in-the-wild using tactile sensing gloves, hand pose tracking gloves, and egocentric glasses. It includes: • 5 hours of real-world data, • 3 hours densely annotated contact-rich interactions, • 2,900 curated interaction clips, • across 800 objects, 14 environments, and 29 grasp types. all open at:

Paul Liang

48,581 görüntüleme • 5 ay önce

Engineers at 1X have developed a 25-degree-of-freedom humanoid robot hand for advanced dexterous manipulation. It uses force sensors in every joint and fingertips to detect pressure, slipping, and automatically adjust its grip. Its lightweight, waterproof design allows the NEO humanoid robot to safely wash its hands after cooking or other messy tasks.
0:32

Engineers at 1X have developed a 25-degree-of-freedom humanoid robot hand for advanced dexterous manipulation. It uses force sensors in every joint and fingertips to detect pressure, slipping, and automatically adjust its grip. Its lightweight, waterproof design allows the NEO humanoid robot to safely wash its hands after cooking or other messy tasks.

Space and Technology

26,780 görüntüleme • 2 ay önce

"We (Democrats) have to provide a vision, and the vision is of a functional democracy. The vision is of a government that is not corrupt. The vision is of a government that is actually focused on solutions for people."
0:25

"We (Democrats) have to provide a vision, and the vision is of a functional democracy. The vision is of a government that is not corrupt. The vision is of a government that is actually focused on solutions for people."

George Conway ⚖️🇺🇸

41,301 görüntüleme • 4 ay önce

More beautiful robot hands. This one from Sharpa The hand uses what they call a Dynamic Tactile Array, meaning vision and touch work together and their SaTA (Spatially-anchored Tactile Awareness) algorithm. Each fingertip includes a mini camera and 1,000+ tactile pixels, hitting 0.005N pressure sensitivity. With 22 DOF and 6D force sensing, it can do everything from eggs to tools.
0:59

More beautiful robot hands. This one from Sharpa The hand uses what they call a Dynamic Tactile Array, meaning vision and touch work together and their SaTA (Spatially-anchored Tactile Awareness) algorithm. Each fingertip includes a mini camera and 1,000+ tactile pixels, hitting 0.005N pressure sensitivity. With 22 DOF and 6D force sensing, it can do everything from eggs to tools.

Rohan Paul

41,775 görüntüleme • 8 ay önce

Chinese robotics company Galaxea Dynamics has introduced a robotic hand called Dexo. It is a four-finger dexterous hand with 17 degrees of freedom for smooth and precise movement. It can sense light touch and handle up to 1 kg with one fingertip, making it useful for both delicate and demanding tasks.
0:24

Chinese robotics company Galaxea Dynamics has introduced a robotic hand called Dexo. It is a four-finger dexterous hand with 17 degrees of freedom for smooth and precise movement. It can sense light touch and handle up to 1 kg with one fingertip, making it useful for both delicate and demanding tasks.

Space and Technology

16,013 görüntüleme • 4 ay önce

Who’s to blame in Houston for the recent struggles of the Rockets? And is a trade needed to solve these issues?
2:48

Who’s to blame in Houston for the recent struggles of the Rockets? And is a trade needed to solve these issues?

Kevin O'Connor

98,817 görüntüleme • 8 ay önce

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)
0:33

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

Chris Paxton

20,486 görüntüleme • 1 yıl önce