Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

A modular data glove built for robot learning. ENSURING’s Magic Glove uses quick-release sensor modules, combining 15 motion-tracking units, 1,000+ tactile sensing points, and real-time 6-axis force data. The modular design is the interesting part here: sensors can be swapped or reconfigured depending on the task, making it a... show more

Eren Chen

10,900 subscribers

33,262 görüntüleme • 2 ay önce •via X (Twitter)

Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

7 Yorum

Eren Chen profil fotoğrafı

Eren Chen2 ay önce

There’s still a fair amount of CGI in the video, but at least part of it is backed by real demos.

Æ profil fotoğrafı

Æ2 ay önce

So close now to the ai powerglove for coding.

Sertac Guven ᯅ profil fotoğrafı

Sertac Guven ᯅ2 ay önce

What ballpark price point are we looking at, for this?

Lukas 'David' profil fotoğrafı

Lukas 'David'2 ay önce

Looks straight out of a sci-fi movie 🤩 The quick-release modules are the real highlight here, way more practical than a fixed setup.

pomider profil fotoğrafı

pomider2 ay önce

Data glove economics finally hit an inflection point. CyberGlove III shipped at $18K with 22 sensors, Meta Manus Prime at $5K with 15. Modular tactile at 1000+ points sits closer to $2K when scaled. VLA training bottleneck was never algorithms - it was per-demo capture cost.

Sn00pD0gg profil fotoğrafı

Sn00pD0gg2 ay önce

Very helpful👍🏻

@𝑀𝑒𝓂𝑒𝓈𝒟𝑒𝓁𝑀𝒶𝓇 profil fotoğrafı

@𝑀𝑒𝓂𝑒𝓈𝒟𝑒𝓁𝑀𝒶𝓇2 ay önce

LFG

Benzer Videolar

What if a dexterous robot could learn a 20+ step chemistry experiment without a single on-robot training demo? Meet TwinDEX: a pair of co-designed, three-finger, nine-DoF dexterous manipulation interface: one wearable for data collection, one for robot deployment. The twinned design shares identical kinematics, contact surfaces, visual appearance, and sensors across collection and deployment — keeping observations and actions aligned end to end. Trained from scratch on only a few hundred wearable demonstrations - with zero on-robot training or intervention data - TwinDEX completed a standardized chemistry experiment involving tool switches, fine force control, and bimanual coordination. Robot-free data showed comparable learning efficiency on the multi-task evaluation, TwinDEX delivered 5.3 times effective throughput than on-robot teleoperation. TwinDEX demonstrates that high-quality robot-free data can fully substitute for on-robot teleoperation data on challenging dexterous tasks — removing the dependency on real-robot hardware that has been the central bottleneck to scaling dexterous manipulation data. This was the proof-it phase. Now comes scale: what emerges at tens of thousands, or millions, of episodes? Watch the demo and read the technical blog: #TwinDEX #Robotics #EmbodiedAI #DexterousManipulation

What if a dexterous robot could learn a 20+ step chemistry experiment without a single on-robot training demo? Meet TwinDEX: a pair of co-designed, three-finger, nine-DoF dexterous manipulation interface: one wearable for data collection, one for robot deployment. The twinned design shares identical kinematics, contact surfaces, visual appearance, and sensors across collection and deployment — keeping observations and actions aligned end to end. Trained from scratch on only a few hundred wearable demonstrations - with zero on-robot training or intervention data - TwinDEX completed a standardized chemistry experiment involving tool switches, fine force control, and bimanual coordination. Robot-free data showed comparable learning efficiency on the multi-task evaluation, TwinDEX delivered 5.3 times effective throughput than on-robot teleoperation. TwinDEX demonstrates that high-quality robot-free data can fully substitute for on-robot teleoperation data on challenging dexterous tasks — removing the dependency on real-robot hardware that has been the central bottleneck to scaling dexterous manipulation data. This was the proof-it phase. Now comes scale: what emerges at tens of thousands, or millions, of episodes? Watch the demo and read the technical blog: #TwinDEX #Robotics #EmbodiedAI #DexterousManipulation

X Square Robot

1,347,091 görüntüleme • 2 ay önce

China's HexaCercle glove captures motion and touch with 408 tactile sensors and multi-axis force sensing. Its joint-angle data maps to AGIBOT’s 10-active/16-total-DoF OmniHand 2025 w/ Tactile.

China's HexaCercle glove captures motion and touch with 408 tactile sensors and multi-axis force sensing. Its joint-angle data maps to AGIBOT’s 10-active/16-total-DoF OmniHand 2025 w/ Tactile.

Eren Chen

28,962 görüntüleme • 2 ay önce

Live from World Robot Conference 2026 in Beijing: ROBOTERA is showcasing teleoperation of the XHAND 1 Pro dexterous hand with MANUS™ gloves. Video 1: a ROBOTERA operator demonstrates high-precision teleoperation. Video 2: a visitor tries the setup without first running a user-specific calibration, yet the teleoperation remains stable, smooth, and responsive. With 21 active degrees of freedom, fully direct-drive actuation, and integrated tactile sensing, XHAND 1 Pro is built for advanced robotic manipulation and embodied intelligence research. MANUS provides high-fidelity, low-latency hand motion data as real-time input for robot control. #WRC2026 #robotics #humanoids #teleoperation #PhysicalAI

Live from World Robot Conference 2026 in Beijing: ROBOTERA is showcasing teleoperation of the XHAND 1 Pro dexterous hand with MANUS™ gloves. Video 1: a ROBOTERA operator demonstrates high-precision teleoperation. Video 2: a visitor tries the setup without first running a user-specific calibration, yet the teleoperation remains stable, smooth, and responsive. With 21 active degrees of freedom, fully direct-drive actuation, and integrated tactile sensing, XHAND 1 Pro is built for advanced robotic manipulation and embodied intelligence research. MANUS provides high-fidelity, low-latency hand motion data as real-time input for robot control. #WRC2026 #robotics #humanoids #teleoperation #PhysicalAI

MANUS™

11,998 görüntüleme • 2 ay önce

A lot of robot learning pipelines treat embodiment mismatch as a software problem. Collect data with one device, retarget the motion to another robot, and try to deal with the differences later. TwinDEX flips that idea around. This system is from a Chinese robotics company called X Square. Instead of starting with a generic data-collection device, the team co-designs a wearable three-finger end-effector and a robotic hand as two sides of the same learning system. They align the things that actually matter during manipulation: joint structure, link proportions, contact geometry, surface properties, visual appearance, sensing, and timing. That matters because dexterous manipulation is much more than matching hand poses. A few millimeters of contact error or a different force relationship can completely change how a task is executed. In the collection evaluation, TwinDEX delivered up to 5.3 times the effective throughput of on-robot teleoperation. It’s a useful reminder that scaling robot learning may not only be about bigger models and more data. Sometimes the hardware collecting the data is part of the data problem itself.

A lot of robot learning pipelines treat embodiment mismatch as a software problem. Collect data with one device, retarget the motion to another robot, and try to deal with the differences later. TwinDEX flips that idea around. This system is from a Chinese robotics company called X Square. Instead of starting with a generic data-collection device, the team co-designs a wearable three-finger end-effector and a robotic hand as two sides of the same learning system. They align the things that actually matter during manipulation: joint structure, link proportions, contact geometry, surface properties, visual appearance, sensing, and timing. That matters because dexterous manipulation is much more than matching hand poses. A few millimeters of contact error or a different force relationship can completely change how a task is executed. In the collection evaluation, TwinDEX delivered up to 5.3 times the effective throughput of on-robot teleoperation. It’s a useful reminder that scaling robot learning may not only be about bigger models and more data. Sometimes the hardware collecting the data is part of the data problem itself.

Alamin

48,112 görüntüleme • 2 ay önce

X Square Robot has introduced TwinDEX: a co-designed pair of three-finger, 9-DoF dexterous hands, one wearable for data collection, one matched robot end effector for deployment. The wearable and the robot hand share the same kinematics, geometry, materials, look, and sensors, so human demos map straight to the robot without retargeting. Human data collection improves throughput 5.3x compared to teleoperation, and a policy can be trained with a few hundred episodes without any robot data.

X Square Robot has introduced TwinDEX: a co-designed pair of three-finger, 9-DoF dexterous hands, one wearable for data collection, one matched robot end effector for deployment. The wearable and the robot hand share the same kinematics, geometry, materials, look, and sensors, so human demos map straight to the robot without retargeting. Human data collection improves throughput 5.3x compared to teleoperation, and a policy can be trained with a few hundred episodes without any robot data.

The Humanoid Hub

277,022 görüntüleme • 2 ay önce

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 görüntüleme • 1 yıl önce

Announcing Mantis UMI, a pair of handheld data collection grippers for teams working on physical AI. Collect accurate bimanual manipulation data in the real world 10× faster than teleoperation. Mantis uses the exact same grippers as Axol, so you can feel what Axol feels. Data collected with Mantis transfers directly to Axol. Mantis runs on Open Source software and is modular so you can chose the tracker that works best you: Vive Tracker, Quest Controllers, and more. Collect it. Train it. Run it on Axol. Available Today 👇

Announcing Mantis UMI, a pair of handheld data collection grippers for teams working on physical AI. Collect accurate bimanual manipulation data in the real world 10× faster than teleoperation. Mantis uses the exact same grippers as Axol, so you can feel what Axol feels. Data collected with Mantis transfers directly to Axol. Mantis runs on Open Source software and is modular so you can chose the tracker that works best you: Vive Tracker, Quest Controllers, and more. Collect it. Train it. Run it on Axol. Available Today 👇

Almond Robotics

31,075 görüntüleme • 2 ay önce

Genesis AI has stepped into the spotlight The company is a SF Bay Area-based full-stack embodied AI startup. The Stack - Foundation model - 20 DoF dexterous hand - Tactile-sensing data-collection glove + Data engine - Custom motor controllers - In-house high-fidelity simulator The Thesis: Manipulation is a system problem, not just an AI problem. The company’s first general-purpose robot is teased as the “ultimate culmination” of the stack. Founders Zhou Xian (CEO) and Théophile Gervet (President), formerly of Mistral, NVIDIA, Google, CMU, MIT, and Stanford. $105 million raised in July 2025. Dual headquarters in San Carlos, CA and Paris.

Genesis AI has stepped into the spotlight The company is a SF Bay Area-based full-stack embodied AI startup. The Stack - Foundation model - 20 DoF dexterous hand - Tactile-sensing data-collection glove + Data engine - Custom motor controllers - In-house high-fidelity simulator The Thesis: Manipulation is a system problem, not just an AI problem. The company’s first general-purpose robot is teased as the “ultimate culmination” of the stack. Founders Zhou Xian (CEO) and Théophile Gervet (President), formerly of Mistral, NVIDIA, Google, CMU, MIT, and Stanford. $105 million raised in July 2025. Dual headquarters in San Carlos, CA and Paris.

The Humanoid Hub

50,111 görüntüleme • 5 ay önce

A robot performing a chemistry experiment might not sound that surprising. What’s interesting about this demo is how the robot was trained. Chinese embodied AI startup X Square Robot's TwinDEX uses a wearable three-finger end-effector to collect demonstrations from a human, rather than requiring the robot to be teleoperated during data collection. The resulting policy handles a 20+ step chemistry workflow: opening reagent bottles, scooping material, operating a pipette, transferring liquids, handling a glass rod, pouring, and coordinating both hands throughout the process. These are very different from simple pick-and-place tasks. There are continuous contacts, tool use, changing grasps, precise positioning, force control, and long-horizon state transitions. The reported experiments used robot-free demonstrations only, with zero on-robot teleoperation data for training. The bigger idea here is that if the data-collection hardware and deployment hardware are designed to be sufficiently consistent, robot-free demonstrations can still retain the information needed for real dexterous manipulation.

A robot performing a chemistry experiment might not sound that surprising. What’s interesting about this demo is how the robot was trained. Chinese embodied AI startup X Square Robot's TwinDEX uses a wearable three-finger end-effector to collect demonstrations from a human, rather than requiring the robot to be teleoperated during data collection. The resulting policy handles a 20+ step chemistry workflow: opening reagent bottles, scooping material, operating a pipette, transferring liquids, handling a glass rod, pouring, and coordinating both hands throughout the process. These are very different from simple pick-and-place tasks. There are continuous contacts, tool use, changing grasps, precise positioning, force control, and long-horizon state transitions. The reported experiments used robot-free demonstrations only, with zero on-robot teleoperation data for training. The bigger idea here is that if the data-collection hardware and deployment hardware are designed to be sufficiently consistent, robot-free demonstrations can still retain the information needed for real dexterous manipulation.

Sharon Riley

18,036 görüntüleme • 2 ay önce

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

Pascal Bornet

186,881 görüntüleme • 7 ay önce

🎉 After one year of robot pretraining, we learned one thing the hard way: data is everything. Today we're excited to release FOMI, a Force-feedback Manipulation Interface for teaching contact-rich manipulation! Inspired by Shuran Song's UMI, FOMI keeps handheld, robot-free data collection and adds what operators have been missing: you can feel what the gripper touches ✋ No force sensors, no motors. Two key results: 🔑 29% more successful demos than UMI in the same time, and 18% more than GELLO in teleoperation 🔑 π0.5 fine-tuned on about an hour of FOMI data per task reaches 78.4% average success on four contact-rich tasks Let's check out our demos 👇

🎉 After one year of robot pretraining, we learned one thing the hard way: data is everything. Today we're excited to release FOMI, a Force-feedback Manipulation Interface for teaching contact-rich manipulation! Inspired by Shuran Song's UMI, FOMI keeps handheld, robot-free data collection and adds what operators have been missing: you can feel what the gripper touches ✋ No force sensors, no motors. Two key results: 🔑 29% more successful demos than UMI in the same time, and 18% more than GELLO in teleoperation 🔑 π0.5 fine-tuned on about an hour of FOMI data per task reaches 78.4% average success on four contact-rich tasks Let's check out our demos 👇

Yinghao Xu

44,806 görüntüleme • 7 gün önce

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes evencheaper with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,358 görüntüleme • 1 yıl önce

We have plenty of human demonstration data for movement. We don’t have enough for touch. Without tactile feedback, foundation models can’t feel if an object is slipping, soft, or fragile during complex manipulation. Matrix Inno is bringing three data-collection gloves to IROS 2026: the RigidCore Tactile+EMF, the Fabric Tactile + EMF, and the Fabric Tactile + UltraLite Motion Glove. The hardware tracks pressure and hand pose simultaneously during human demos, mapping exactly where contact happens and how it shifts in real time. Wearables like this are how we finally get real, continuous contact data at scale. Source: Matrix Inno MX MATRIX

We have plenty of human demonstration data for movement. We don’t have enough for touch. Without tactile feedback, foundation models can’t feel if an object is slipping, soft, or fragile during complex manipulation. Matrix Inno is bringing three data-collection gloves to IROS 2026: the RigidCore Tactile+EMF, the Fabric Tactile + EMF, and the Fabric Tactile + UltraLite Motion Glove. The hardware tracks pressure and hand pose simultaneously during human demos, mapping exactly where contact happens and how it shifts in real time. Wearables like this are how we finally get real, continuous contact data at scale. Source: Matrix Inno MX MATRIX

RoboHub🤖

12,372 görüntüleme • 1 ay önce

A high-quality haptic control for robots as a weekend project? All for under $600. [📍 Save Github & arXiv for later] A team built DOGlove, an open-source haptic glove for robot teleoperation that you can assemble yourself for under $600. It is built for real control, not demos. • Tracks 21 degrees of freedom with a custom joint design • Provides cable-driven force feedback plus fingertip haptics • Can be used to collect data for imitation learning Affordable haptics could make teleoperation more reliable and improve the quality of data we use to train physical AI. Project link: Credit: DOGlove team, shared by Han Zhang —- Weekly robotics and AI insights. Subscribe free:

A high-quality haptic control for robots as a weekend project? All for under $600. [📍 Save Github & arXiv for later] A team built DOGlove, an open-source haptic glove for robot teleoperation that you can assemble yourself for under $600. It is built for real control, not demos. • Tracks 21 degrees of freedom with a custom joint design • Provides cable-driven force feedback plus fingertip haptics • Can be used to collect data for imitation learning Affordable haptics could make teleoperation more reliable and improve the quality of data we use to train physical AI. Project link: Credit: DOGlove team, shared by Han Zhang —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

18,290 görüntüleme • 9 ay önce

Impressive on every dimension! Genesis AI has announced GENE-26.5, its first robotic foundation model, aimed at achieving human-level dexterous manipulation: - Single model, shared weights, handles egg cracking, lab pipetting, bimanual Rubik's Cube, smoothie making, and wire harnessing. Most tasks: under 1 hour of task-specific data, 1x real-world speed. - Genesis Hand 1.0: human-size, 20 active DoF, soft-contact skin. Paired glove gives a 1:1:1 mapping (glove, human hand, robot hand). 100x cheaper than teleop, 5x more data-efficient internally. - Data engine pulls from three sources: glove data, egocentric (head-cam) video, and large-scale internet video. The 1:1 hand-to-human match closes the embodiment gap, letting Genesis use video data more effectively than rivals. - Flow matching across vision, tactile, proprioception, and language inputs. Per Genesis, scaling data and compute directly improves zero-shot

Impressive on every dimension! Genesis AI has announced GENE-26.5, its first robotic foundation model, aimed at achieving human-level dexterous manipulation: - Single model, shared weights, handles egg cracking, lab pipetting, bimanual Rubik's Cube, smoothie making, and wire harnessing. Most tasks: under 1 hour of task-specific data, 1x real-world speed. - Genesis Hand 1.0: human-size, 20 active DoF, soft-contact skin. Paired glove gives a 1:1:1 mapping (glove, human hand, robot hand). 100x cheaper than teleop, 5x more data-efficient internally. - Data engine pulls from three sources: glove data, egocentric (head-cam) video, and large-scale internet video. The 1:1 hand-to-human match closes the embodiment gap, letting Genesis use video data more effectively than rivals. - Flow matching across vision, tactile, proprioception, and language inputs. Per Genesis, scaling data and compute directly improves zero-shot

The Humanoid Hub

96,023 görüntüleme • 5 ay önce

TidyBot++ is an open-source mobile manipulator optimized for household tasks. The robot can be teleoperated using a mobile phone interface, enabling data collection for imitation learning.

TidyBot++ is an open-source mobile manipulator optimized for household tasks. The robot can be teleoperated using a mobile phone interface, enabling data collection for imitation learning.

The Humanoid Hub

30,105 görüntüleme • 1 yıl önce

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n

Zhi (Leo) Wang

53,753 görüntüleme • 4 ay önce

I was really impressed by the UMI gripper (Cheng Chi et al.), but a key limitation is that **force-related data wasn’t captured**: humans feel haptic feedback through the mechanical springs, but the robot couldn’t leverage that info, limiting the data’s value for fine-grained manipulation tasks. Led by my amazing students Yolanda Zhu and Binghao Huang, we designed a **portable visuo-tactile gripper** by integrating our dense, flexible tactile arrays with the UMI gripper to enable large-scale in-the-wild data collection. 🔗 We demonstrate **cross-modal representation learning** and **downstream policy learning** on tasks requiring in-hand state estimation (e.g., test tube reorientation) and fine-grained force sensing (e.g., pipette fluid transfer). Key takeaways: - Our flexible tactile arrays store the rich haptic information humans perceive as dense tactile signals. - Portability and robustness are key for in-the-wild data collection; our portable gripper is compact, lightweight, and durable. - Touch provides precise, robust measurements of in-hand object pose, invariant to lighting and viewpoint. - Cross-modal pretraining on large-scale in-the-wild data significantly improves policy robustness and sample efficiency (as shown many times before — and verified again here!). Also check out our previous investigations of dense, flexible tactile grids for understanding human-robot-environment interactions: - Dense tactile glove (Nature ’19): - 3D-ViTac (CoRL ’24):

I was really impressed by the UMI gripper (Cheng Chi et al.), but a key limitation is that force-related data wasn’t captured: humans feel haptic feedback through the mechanical springs, but the robot couldn’t leverage that info, limiting the data’s value for fine-grained manipulation tasks. Led by my amazing students Yolanda Zhu and Binghao Huang, we designed a portable visuo-tactile gripper by integrating our dense, flexible tactile arrays with the UMI gripper to enable large-scale in-the-wild data collection. 🔗 We demonstrate cross-modal representation learning and downstream policy learning on tasks requiring in-hand state estimation (e.g., test tube reorientation) and fine-grained force sensing (e.g., pipette fluid transfer). Key takeaways: - Our flexible tactile arrays store the rich haptic information humans perceive as dense tactile signals. - Portability and robustness are key for in-the-wild data collection; our portable gripper is compact, lightweight, and durable. - Touch provides precise, robust measurements of in-hand object pose, invariant to lighting and viewpoint. - Cross-modal pretraining on large-scale in-the-wild data significantly improves policy robustness and sample efficiency (as shown many times before — and verified again here!). Also check out our previous investigations of dense, flexible tactile grids for understanding human-robot-environment interactions: - Dense tactile glove (Nature ’19): - 3D-ViTac (CoRL ’24):

Yunzhu Li

13,510 görüntüleme • 1 yıl önce

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.

AI at Meta

453,860 görüntüleme • 2 yıl önce

📢📢📢 Excited to release ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR25). 🤏🤙✌️With ManipTrans, we can transfer dexterous manipulation skills into robotic hands in simulation and deploy them on a real robot, using a residual policy learned for dex manipulation. 🤖🤖🤖The video below illustrates how the MoCap data can be transferred to Inspire, Shadow, Xhand, Allegro, and Mano. With ManipTrans, we can scale up dex manip data greatly with minimal effort. For more details, please check our -webpage: -code: -huggingface:

📢📢📢 Excited to release ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR25). 🤏🤙✌️With ManipTrans, we can transfer dexterous manipulation skills into robotic hands in simulation and deploy them on a real robot, using a residual policy learned for dex manipulation. 🤖🤖🤖The video below illustrates how the MoCap data can be transferred to Inspire, Shadow, Xhand, Allegro, and Mano. With ManipTrans, we can scale up dex manip data greatly with minimal effort. For more details, please check our -webpage: -code: -huggingface:

Siyuan Huang

20,983 görüntüleme • 1 yıl önce