Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Vision alone isn’t enough to solve dexterous manipulation. The sense of touch is needed. UMI-FT uses a custom PCB to provide 6-axis force-torque sensing at each finger for under $10.

Simon Kalouche

10,649 subscribers

20,731 просмотров • 7 месяцев назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 11

Фото профиля Simon Kalouche

Simon Kalouche7 месяцев назад

UMI-FT Project website: CoinFT Project website:

Фото профиля Embedr

Embedr7 месяцев назад

this could seriously democratize robotics research. no more selling a kidney for decent tactile feedback

Фото профиля MechatronicsEngineer

MechatronicsEngineer7 месяцев назад

tactile revolution begins

Фото профиля Devesh Vyas

Devesh Vyas7 месяцев назад

Yeah, robots needs data about torque, force, and other parameters to gauge the situation and perform tasks aptly. Vision alone ain’t enough. Though it could teach a few things, but still data from actuators and other sensors would be the moat going forward

Фото профиля Akshay Kumar

Akshay Kumar7 месяцев назад

I keep wondering why the biggest labs building VLAs have barely scraped the surface of tactile sensing and joint torque sensing modalities yet! Perhaps a data issue?

Фото профиля Peter Kazanjy

Peter Kazanjy7 месяцев назад

Sick demo. Going to be used in Nimble?

Фото профиля Sudhir Pratap Yadav

Sudhir Pratap Yadav7 месяцев назад

Can you explain little bit what is happening in first video. I can its responding to the push but like while maintaining ee position? Like orientation chnage is allowed etc? What kind of controller is it, is it OSC?

Фото профиля Lidoor L. Joseph

Lidoor L. Joseph7 месяцев назад

This

Фото профиля Gabriele Tinelli

Gabriele Tinelli7 месяцев назад

Do you think this scales beyond development environments? Will production robots still need expensive FT?

Фото профиля milan 🌞

milan 🌞7 месяцев назад

Awesome stuff. I agree, touch is absolutely necessary.

Фото профиля adamyathegreat

adamyathegreat7 месяцев назад

Cool

Похожие видео

Force-sensing fingers! 🧤 Stanford researchers just released UMI-FT, a handheld data collection platform that puts compact six-axis force/torque sensors on each finger, enabling finger-level wrench measurements alongside RGB, depth, and pose data. Many manipulation tasks require careful force modulation: too little force and the task fails, too much and you cause damage. But commercial force/torque sensors are expensive, bulky, and fragile, which has limited large-scale force-aware policy learning. UMI-FT changes the economics. The platform uses an iPhone for RGB vision, ultrawide RGB, depth, and pose via ARKit, with each finger sensorized using a CoinFT sensor to capture per-finger wrench information during manipulation. This multimodal data trains an adaptive compliance policy that predicts position targets, grasp force, and stiffness for execution on standard compliance controllers. The learned policy runs slowest and generates reference targets, while model-based compliance and force controllers provide delicate 6D compliance control and real-time force modulation. They tested on three contact-rich, force-sensitive tasks: whiteboard wiping (locate eraser, grasp, wipe until clean), skewering zucchini (grasp slice firmly, push onto stick until punctured), and lightbulb insertion (grasp bulb, align bayonet pin with socket slit, insert while overcoming spring force, rotate to light up). The results are clear. Policies without compliance struggle to modulate contact force and trigger safety faults from excessive force. Policies without force sensing fail to grasp unseen objects or resist reaction forces, causing slippage. Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Force-sensing fingers! 🧤 Stanford researchers just released UMI-FT, a handheld data collection platform that puts compact six-axis force/torque sensors on each finger, enabling finger-level wrench measurements alongside RGB, depth, and pose data. Many manipulation tasks require careful force modulation: too little force and the task fails, too much and you cause damage. But commercial force/torque sensors are expensive, bulky, and fragile, which has limited large-scale force-aware policy learning. UMI-FT changes the economics. The platform uses an iPhone for RGB vision, ultrawide RGB, depth, and pose via ARKit, with each finger sensorized using a CoinFT sensor to capture per-finger wrench information during manipulation. This multimodal data trains an adaptive compliance policy that predicts position targets, grasp force, and stiffness for execution on standard compliance controllers. The learned policy runs slowest and generates reference targets, while model-based compliance and force controllers provide delicate 6D compliance control and real-time force modulation. They tested on three contact-rich, force-sensitive tasks: whiteboard wiping (locate eraser, grasp, wipe until clean), skewering zucchini (grasp slice firmly, push onto stick until punctured), and lightbulb insertion (grasp bulb, align bayonet pin with socket slit, insert while overcoming spring force, rotate to light up). The results are clear. Policies without compliance struggle to modulate contact force and trigger safety faults from excessive force. Policies without force sensing fail to grasp unseen objects or resist reaction forces, causing slippage. Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

12,822 просмотров • 7 месяцев назад

A modular data glove built for robot learning. ENSURING’s Magic Glove uses quick-release sensor modules, combining 15 motion-tracking units, 1,000+ tactile sensing points, and real-time 6-axis force data. The modular design is the interesting part here: sensors can be swapped or reconfigured depending on the task, making it a more flexible interface for teleoperation, dexterous manipulation, and embodied AI data collection.

A modular data glove built for robot learning. ENSURING’s Magic Glove uses quick-release sensor modules, combining 15 motion-tracking units, 1,000+ tactile sensing points, and real-time 6-axis force data. The modular design is the interesting part here: sensors can be swapped or reconfigured depending on the task, making it a more flexible interface for teleoperation, dexterous manipulation, and embodied AI data collection.

Eren Chen

33,006 просмотров • 1 месяц назад

Tactile feedback is one of the most important modalities in manipulation, but has been underutilized in dexterous hands. T-Dex is a framework for learning dexterous policies from tactile play data, beating vision and torque-based methods by 1.7x. 🧵👇

Tactile feedback is one of the most important modalities in manipulation, but has been underutilized in dexterous hands. T-Dex is a framework for learning dexterous policies from tactile play data, beating vision and torque-based methods by 1.7x. 🧵👇

Lerrel Pinto

84,611 просмотров • 3 лет назад

Open-source dexterous hands with fingertip sensors! 🪬 ORCA Dexterity just released three dexterous hand models. Their mission: is to democratize robotic hand dexterity. They're sharing progress on exceptional, low-cost hardware and a software layer from low-level control to robotic hand learning. This is the open-source approach to dexterous manipulation. It has 83 taxels per finger with 0.1 N force detection which pretty is impressive for an open-source design. Tactile sensing is critical for dexterous manipulation, knowing contact forces enables gentle grasping, slip detection, and force-controlled assembly. Also 700g weight for the lite version makes it practical for mounting on robot arms without exceeding payload limits. Lower weight means faster movements and lower torque requirements. Open hardware accelerates robotics by letting researchers and builders modify designs for their specific needs without starting from scratch! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Open-source dexterous hands with fingertip sensors! 🪬 ORCA Dexterity just released three dexterous hand models. Their mission: is to democratize robotic hand dexterity. They're sharing progress on exceptional, low-cost hardware and a software layer from low-level control to robotic hand learning. This is the open-source approach to dexterous manipulation. It has 83 taxels per finger with 0.1 N force detection which pretty is impressive for an open-source design. Tactile sensing is critical for dexterous manipulation, knowing contact forces enables gentle grasping, slip detection, and force-controlled assembly. Also 700g weight for the lite version makes it practical for mounting on robot arms without exceeding payload limits. Lower weight means faster movements and lower torque requirements. Open hardware accelerates robotics by letting researchers and builders modify designs for their specific needs without starting from scratch! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

38,249 просмотров • 6 месяцев назад

We just released TAVI -- a robotics framework that combines touch and vision to solve challenging dexterous tasks in under 1 hour. The key? Use human demonstrations to initialize a policy, followed by tactile-based online learning with vision-based rewards. Details in🧵(1/7)

We just released TAVI -- a robotics framework that combines touch and vision to solve challenging dexterous tasks in under 1 hour. The key? Use human demonstrations to initialize a policy, followed by tactile-based online learning with vision-based rewards. Details in🧵(1/7)

Lerrel Pinto

138,931 просмотров • 3 лет назад

Touch alone isn’t enough. 🖐️ For robotics, tactile intelligence truly levels up when touch gains spatial meaning. At #ICRA2026, we dove into the core concept behind SaTA: Spatially-anchored Tactile Awareness for robust, dexterous manipulation. Read it here: The challenge is fundamental: a robot shouldn’t just register a touch, it needs to understand exactly where that contact occurs relative to its fingers, joints, and overall hand structure. This is the missing link that turns raw data into precise, real-time adjustments during manipulation. Why does this matter? Because the most complex part of any manipulation task happens when vision is at its least reliable, that final millimeter before insertion, sliding, gripping, or fine-tuning. At Sharpa, this is exactly why we’re building tactile hands and tactile AI in tandem. 🚀 #Sharpa #Robotics #EmbodiedAI #TactileIntelligence #ICRA2026 #DexterousManipulation 📷

Touch alone isn’t enough. 🖐️ For robotics, tactile intelligence truly levels up when touch gains spatial meaning. At #ICRA2026, we dove into the core concept behind SaTA: Spatially-anchored Tactile Awareness for robust, dexterous manipulation. Read it here: The challenge is fundamental: a robot shouldn’t just register a touch, it needs to understand exactly where that contact occurs relative to its fingers, joints, and overall hand structure. This is the missing link that turns raw data into precise, real-time adjustments during manipulation. Why does this matter? Because the most complex part of any manipulation task happens when vision is at its least reliable, that final millimeter before insertion, sliding, gripping, or fine-tuning. At Sharpa, this is exactly why we’re building tactile hands and tactile AI in tandem. 🚀 #Sharpa #Robotics #EmbodiedAI #TactileIntelligence #ICRA2026 #DexterousManipulation 📷

Sharpa

15,168 просмотров • 3 месяцев назад

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n

Zhi (Leo) Wang

52,211 просмотров • 2 месяцев назад

From day one, mimic has been focused on a single goal: general-purpose dexterous manipulation. Today we're proud to announce the mimic hand M1 and the mimic wearable U1. We believe the only way to solve dexterous manipulation at scale is by going full-stack at the frontier of physical AI, building every layer ourselves around one fixed point, the human hand. The M1 is a highly backdrivable, tendon-driven hand that covers the full range of human capability, from heavy payloads to fine manipulation.

From day one, mimic has been focused on a single goal: general-purpose dexterous manipulation. Today we're proud to announce the mimic hand M1 and the mimic wearable U1. We believe the only way to solve dexterous manipulation at scale is by going full-stack at the frontier of physical AI, building every layer ourselves around one fixed point, the human hand. The M1 is a highly backdrivable, tendon-driven hand that covers the full range of human capability, from heavy payloads to fine manipulation.

mimic

92,695 просмотров • 2 месяцев назад

We went to Wuji's Shenzhen office to get a grip on their dexterous hand design. Each finger joint is driven directly by a tiny motor paired with a worm gear that redirects the axis of rotation. We spoke with a senior team member there to get more insights. Lena RoboHub🤖 Thanks WUJI TECH for the tour!

We went to Wuji's Shenzhen office to get a grip on their dexterous hand design. Each finger joint is driven directly by a tiny motor paired with a worm gear that redirects the axis of rotation. We spoke with a senior team member there to get more insights. Lena RoboHub🤖 Thanks WUJI TECH for the tour!

The Humanoid Hub

29,546 просмотров • 2 месяцев назад

Jamie Dimon CEO of JP Morgan on raising taxes: “Giving Washington another Trillion isn’t going to solve anything” He uses the word swamp to describe all the special interests. Moderator tried to end his observation. Jaimie is sensing the change of tolerance outside that room for continuing business as usual.

Jamie Dimon CEO of JP Morgan on raising taxes: “Giving Washington another Trillion isn’t going to solve anything” He uses the word swamp to describe all the special interests. Moderator tried to end his observation. Jaimie is sensing the change of tolerance outside that room for continuing business as usual.

Edward Dowd

85,671 просмотров • 7 месяцев назад

A dexterous hand capable of operating an iPhone Impressive, with nimble fingertip control. INSPIRE ROBOTS from Beijing has released a new dexterous hand featuring tactile and force sensing, and equipped with a motion library, making it ready to use right out of the box. Especially noteworthy is the modular finger design, allowing users to easily replace a damaged finger. With 13 degrees of freedom and a single-hand load capacity of 30kg, it can be used in general applications such as factory component handling, precision assembly, and warehouse logistics sorting. Notably, this company has already sold over 10,000 of these dexterous hands.

A dexterous hand capable of operating an iPhone Impressive, with nimble fingertip control. INSPIRE ROBOTS from Beijing has released a new dexterous hand featuring tactile and force sensing, and equipped with a motion library, making it ready to use right out of the box. Especially noteworthy is the modular finger design, allowing users to easily replace a damaged finger. With 13 degrees of freedom and a single-hand load capacity of 30kg, it can be used in general applications such as factory component handling, precision assembly, and warehouse logistics sorting. Notably, this company has already sold over 10,000 of these dexterous hands.

CyberRobo

14,196 просмотров • 8 месяцев назад

Most of today's AI can see the world, but it doesn’t **feel** it. Capturing the sense of touch is crucial for dexterous robotic manipulation, user modeling, and understanding physical interactions. Introducing OpenTouch: bringing full-hand tactile sensing into real-world AI🖐️ OpenTouch is collected in-the-wild using tactile sensing gloves, hand pose tracking gloves, and egocentric glasses. It includes: • 5 hours of real-world data, • 3 hours densely annotated contact-rich interactions, • 2,900 curated interaction clips, • across 800 objects, 14 environments, and 29 grasp types. all open at:

Most of today's AI can see the world, but it doesn’t feel it. Capturing the sense of touch is crucial for dexterous robotic manipulation, user modeling, and understanding physical interactions. Introducing OpenTouch: bringing full-hand tactile sensing into real-world AI🖐️ OpenTouch is collected in-the-wild using tactile sensing gloves, hand pose tracking gloves, and egocentric glasses. It includes: • 5 hours of real-world data, • 3 hours densely annotated contact-rich interactions, • 2,900 curated interaction clips, • across 800 objects, 14 environments, and 29 grasp types. all open at:

Paul Liang

48,581 просмотров • 5 месяцев назад

Engineers at 1X have developed a 25-degree-of-freedom humanoid robot hand for advanced dexterous manipulation. It uses force sensors in every joint and fingertips to detect pressure, slipping, and automatically adjust its grip. Its lightweight, waterproof design allows the NEO humanoid robot to safely wash its hands after cooking or other messy tasks.

Engineers at 1X have developed a 25-degree-of-freedom humanoid robot hand for advanced dexterous manipulation. It uses force sensors in every joint and fingertips to detect pressure, slipping, and automatically adjust its grip. Its lightweight, waterproof design allows the NEO humanoid robot to safely wash its hands after cooking or other messy tasks.

Space and Technology

26,780 просмотров • 2 месяцев назад

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:

Haoran Geng

95,924 просмотров • 1 год назад

"We (Democrats) have to provide a vision, and the vision is of a functional democracy. The vision is of a government that is not corrupt. The vision is of a government that is actually focused on solutions for people."

"We (Democrats) have to provide a vision, and the vision is of a functional democracy. The vision is of a government that is not corrupt. The vision is of a government that is actually focused on solutions for people."

George Conway ⚖️🇺🇸

41,301 просмотров • 4 месяцев назад

More beautiful robot hands. This one from Sharpa The hand uses what they call a Dynamic Tactile Array, meaning vision and touch work together and their SaTA (Spatially-anchored Tactile Awareness) algorithm. Each fingertip includes a mini camera and 1,000+ tactile pixels, hitting 0.005N pressure sensitivity. With 22 DOF and 6D force sensing, it can do everything from eggs to tools.

More beautiful robot hands. This one from Sharpa The hand uses what they call a Dynamic Tactile Array, meaning vision and touch work together and their SaTA (Spatially-anchored Tactile Awareness) algorithm. Each fingertip includes a mini camera and 1,000+ tactile pixels, hitting 0.005N pressure sensitivity. With 22 DOF and 6D force sensing, it can do everything from eggs to tools.

Rohan Paul

41,775 просмотров • 8 месяцев назад

Chinese robotics company Galaxea Dynamics has introduced a robotic hand called Dexo. It is a four-finger dexterous hand with 17 degrees of freedom for smooth and precise movement. It can sense light touch and handle up to 1 kg with one fingertip, making it useful for both delicate and demanding tasks.

Chinese robotics company Galaxea Dynamics has introduced a robotic hand called Dexo. It is a four-finger dexterous hand with 17 degrees of freedom for smooth and precise movement. It can sense light touch and handle up to 1 kg with one fingertip, making it useful for both delicate and demanding tasks.

Space and Technology

16,013 просмотров • 4 месяцев назад

Who’s to blame in Houston for the recent struggles of the Rockets? And is a trade needed to solve these issues?

Who’s to blame in Houston for the recent struggles of the Rockets? And is a trade needed to solve these issues?

Kevin O'Connor

98,817 просмотров • 8 месяцев назад

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

Chris Paxton

20,486 просмотров • 1 год назад