Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

We have plenty of human demonstration data for movement. We don’t have enough for touch. Without tactile feedback, foundation models can’t feel if an object is slipping, soft, or fragile during complex manipulation. Matrix Inno is bringing three data-collection gloves to IROS 2026: the RigidCore Tactile+EMF, the Fabric Tactile... show more

RoboHub🤖

22,351 subscribers

12,372 Aufrufe • vor 18 Tagen •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

4 Kommentare

Profilbild von Zhen

Zhenvor 16 Tagen

touch is the missing modality everyone hand-waves about and nobody collects data for. vision got ImageNet, manipulation needs its own data flywheel, and gloves capturing pressure plus pose during human demos is the most practical path to it. whoever owns the tactile dataset owns the next leap in dexterity.

Profilbild von Steven Cheng

Steven Chengvor 18 Tagen

Guess my robot arm’s “feel” is just a 404 error right now. Cool to see gloves fixing the touch gap, though.

Profilbild von RoboHub🤖

RoboHub🤖vor 17 Tagen

Time to give that robot arm a dexterous hand and put it to work 😂

Profilbild von Omnimes — Smart AI MES

Omnimes — Smart AI MESvor 17 Tagen

The real bottleneck with tactile gloves isn't the missing data, it's data quality at low force. Recent academic work shows hand pose itself generates false force readings in flexible sensors, which raises the minimum detectable force and corrupts exactly the subtle slip moments you're describing.

Ähnliche Videos

Touch alone isn’t enough. 🖐️ For robotics, tactile intelligence truly levels up when touch gains spatial meaning. At #ICRA2026, we dove into the core concept behind SaTA: Spatially-anchored Tactile Awareness for robust, dexterous manipulation. Read it here: The challenge is fundamental: a robot shouldn’t just register a touch, it needs to understand exactly where that contact occurs relative to its fingers, joints, and overall hand structure. This is the missing link that turns raw data into precise, real-time adjustments during manipulation. Why does this matter? Because the most complex part of any manipulation task happens when vision is at its least reliable, that final millimeter before insertion, sliding, gripping, or fine-tuning. At Sharpa, this is exactly why we’re building tactile hands and tactile AI in tandem. 🚀 #Sharpa #Robotics #EmbodiedAI #TactileIntelligence #ICRA2026 #DexterousManipulation 📷

Touch alone isn’t enough. 🖐️ For robotics, tactile intelligence truly levels up when touch gains spatial meaning. At #ICRA2026, we dove into the core concept behind SaTA: Spatially-anchored Tactile Awareness for robust, dexterous manipulation. Read it here: The challenge is fundamental: a robot shouldn’t just register a touch, it needs to understand exactly where that contact occurs relative to its fingers, joints, and overall hand structure. This is the missing link that turns raw data into precise, real-time adjustments during manipulation. Why does this matter? Because the most complex part of any manipulation task happens when vision is at its least reliable, that final millimeter before insertion, sliding, gripping, or fine-tuning. At Sharpa, this is exactly why we’re building tactile hands and tactile AI in tandem. 🚀 #Sharpa #Robotics #EmbodiedAI #TactileIntelligence #ICRA2026 #DexterousManipulation 📷

Sharpa

15,402 Aufrufe • vor 4 Monaten

Most of today's AI can see the world, but it doesn’t **feel** it. Capturing the sense of touch is crucial for dexterous robotic manipulation, user modeling, and understanding physical interactions. Introducing OpenTouch: bringing full-hand tactile sensing into real-world AI🖐️ OpenTouch is collected in-the-wild using tactile sensing gloves, hand pose tracking gloves, and egocentric glasses. It includes: • 5 hours of real-world data, • 3 hours densely annotated contact-rich interactions, • 2,900 curated interaction clips, • across 800 objects, 14 environments, and 29 grasp types. all open at:

Most of today's AI can see the world, but it doesn’t feel it. Capturing the sense of touch is crucial for dexterous robotic manipulation, user modeling, and understanding physical interactions. Introducing OpenTouch: bringing full-hand tactile sensing into real-world AI🖐️ OpenTouch is collected in-the-wild using tactile sensing gloves, hand pose tracking gloves, and egocentric glasses. It includes: • 5 hours of real-world data, • 3 hours densely annotated contact-rich interactions, • 2,900 curated interaction clips, • across 800 objects, 14 environments, and 29 grasp types. all open at:

Paul Liang

49,323 Aufrufe • vor 6 Monaten

I was really impressed by the UMI gripper (Cheng Chi et al.), but a key limitation is that **force-related data wasn’t captured**: humans feel haptic feedback through the mechanical springs, but the robot couldn’t leverage that info, limiting the data’s value for fine-grained manipulation tasks. Led by my amazing students Yolanda Zhu and Binghao Huang, we designed a **portable visuo-tactile gripper** by integrating our dense, flexible tactile arrays with the UMI gripper to enable large-scale in-the-wild data collection. 🔗 We demonstrate **cross-modal representation learning** and **downstream policy learning** on tasks requiring in-hand state estimation (e.g., test tube reorientation) and fine-grained force sensing (e.g., pipette fluid transfer). Key takeaways: - Our flexible tactile arrays store the rich haptic information humans perceive as dense tactile signals. - Portability and robustness are key for in-the-wild data collection; our portable gripper is compact, lightweight, and durable. - Touch provides precise, robust measurements of in-hand object pose, invariant to lighting and viewpoint. - Cross-modal pretraining on large-scale in-the-wild data significantly improves policy robustness and sample efficiency (as shown many times before — and verified again here!). Also check out our previous investigations of dense, flexible tactile grids for understanding human-robot-environment interactions: - Dense tactile glove (Nature ’19): - 3D-ViTac (CoRL ’24):

I was really impressed by the UMI gripper (Cheng Chi et al.), but a key limitation is that force-related data wasn’t captured: humans feel haptic feedback through the mechanical springs, but the robot couldn’t leverage that info, limiting the data’s value for fine-grained manipulation tasks. Led by my amazing students Yolanda Zhu and Binghao Huang, we designed a portable visuo-tactile gripper by integrating our dense, flexible tactile arrays with the UMI gripper to enable large-scale in-the-wild data collection. 🔗 We demonstrate cross-modal representation learning and downstream policy learning on tasks requiring in-hand state estimation (e.g., test tube reorientation) and fine-grained force sensing (e.g., pipette fluid transfer). Key takeaways: - Our flexible tactile arrays store the rich haptic information humans perceive as dense tactile signals. - Portability and robustness are key for in-the-wild data collection; our portable gripper is compact, lightweight, and durable. - Touch provides precise, robust measurements of in-hand object pose, invariant to lighting and viewpoint. - Cross-modal pretraining on large-scale in-the-wild data significantly improves policy robustness and sample efficiency (as shown many times before — and verified again here!). Also check out our previous investigations of dense, flexible tactile grids for understanding human-robot-environment interactions: - Dense tactile glove (Nature ’19): - 3D-ViTac (CoRL ’24):

Yunzhu Li

13,510 Aufrufe • vor 1 Jahr

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n

Zhi (Leo) Wang

53,413 Aufrufe • vor 3 Monaten

Manipulation happens through surfaces. To understand contact-rich dexterous interaction, motion alone is not enough. We also need to know surface properties and contact state. Excited to share ART-Glove, an articulated tactile glove that captures contact-grounded information while preserving human dexterity. It provides: - Known Geometry: 16 rigid functional surfaces - Surface Motion: 22 anatomically aligned joints - Tactile Contact: 2048 piezoresistive taxels Huge thanks to my advisor Ding Ding Zhao, and to Yuxiang Yuxiang Yang, Maria Maria Bauza Villalonga, Marissa, and Peide Peide Huang for the valuable advice and discussions. Paper:

Manipulation happens through surfaces. To understand contact-rich dexterous interaction, motion alone is not enough. We also need to know surface properties and contact state. Excited to share ART-Glove, an articulated tactile glove that captures contact-grounded information while preserving human dexterity. It provides: - Known Geometry: 16 rigid functional surfaces - Surface Motion: 22 anatomically aligned joints - Tactile Contact: 2048 piezoresistive taxels Huge thanks to my advisor Ding Ding Zhao, and to Yuxiang Yuxiang Yang, Maria Maria Bauza Villalonga, Marissa, and Peide Peide Huang for the valuable advice and discussions. Paper:

Changyi Lin

88,692 Aufrufe • vor 3 Monaten

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.

AI at Meta

453,860 Aufrufe • vor 1 Jahr

Impressive on every dimension! Genesis AI has announced GENE-26.5, its first robotic foundation model, aimed at achieving human-level dexterous manipulation: - Single model, shared weights, handles egg cracking, lab pipetting, bimanual Rubik's Cube, smoothie making, and wire harnessing. Most tasks: under 1 hour of task-specific data, 1x real-world speed. - Genesis Hand 1.0: human-size, 20 active DoF, soft-contact skin. Paired glove gives a 1:1:1 mapping (glove, human hand, robot hand). 100x cheaper than teleop, 5x more data-efficient internally. - Data engine pulls from three sources: glove data, egocentric (head-cam) video, and large-scale internet video. The 1:1 hand-to-human match closes the embodiment gap, letting Genesis use video data more effectively than rivals. - Flow matching across vision, tactile, proprioception, and language inputs. Per Genesis, scaling data and compute directly improves zero-shot

Impressive on every dimension! Genesis AI has announced GENE-26.5, its first robotic foundation model, aimed at achieving human-level dexterous manipulation: - Single model, shared weights, handles egg cracking, lab pipetting, bimanual Rubik's Cube, smoothie making, and wire harnessing. Most tasks: under 1 hour of task-specific data, 1x real-world speed. - Genesis Hand 1.0: human-size, 20 active DoF, soft-contact skin. Paired glove gives a 1:1:1 mapping (glove, human hand, robot hand). 100x cheaper than teleop, 5x more data-efficient internally. - Data engine pulls from three sources: glove data, egocentric (head-cam) video, and large-scale internet video. The 1:1 hand-to-human match closes the embodiment gap, letting Genesis use video data more effectively than rivals. - Flow matching across vision, tactile, proprioception, and language inputs. Per Genesis, scaling data and compute directly improves zero-shot

The Humanoid Hub

95,894 Aufrufe • vor 5 Monaten

The sense of touch is the most criminally under-explored modality in robotics. Imagine doing sleight of hand wearing thick oven mitts. That's exactly how a robot feels today if it were alive. A magnetic piece snapping into place, a paper cup peeling out of a stack, a USB negotiating its way into the port - all invisible to the camera. Learning how to feel must be a full-stack co-designed effort. We are open-sourcing a principled methodology called "T-Rex": 1. Tactile as first-class citizen of the model. Our mixture-of-transformer runs two clocks asynchronously: a slow visuomotor expert plans the motion, and a fast tactile expert refines it in real time with high-frequency corrections at 4 "touch ticks" per vision tick. Forces change faster than frames arrive, so the architecture had to as well. 2. Open data. The largest tactile dataset ever released to our knowledge: a 50-hour (~5,500 episodes) high-quality, carefully synchronized robot play corpus, collected on SOTA tactile hand hardware with 22 degrees of freedom. Available today on HuggingFace! 3. Training recipe: T-Rex extends our prior work, EgoScale. Human egocentric videos for pretraining, a diverse dose of tactile robot play for mid-training. Our experiments show this bridges contact-free pretraining to contact-rich manipulation remarkably well. Pixels are cheap and everywhere, but they run out of steam at the moment of contact. Tactile will carry the last mile. The next scaling curve will be measured in hours of touch. T-Rex is a great collaboration between NVIDIA and Berkeley: 🧵

The sense of touch is the most criminally under-explored modality in robotics. Imagine doing sleight of hand wearing thick oven mitts. That's exactly how a robot feels today if it were alive. A magnetic piece snapping into place, a paper cup peeling out of a stack, a USB negotiating its way into the port - all invisible to the camera. Learning how to feel must be a full-stack co-designed effort. We are open-sourcing a principled methodology called "T-Rex": 1. Tactile as first-class citizen of the model. Our mixture-of-transformer runs two clocks asynchronously: a slow visuomotor expert plans the motion, and a fast tactile expert refines it in real time with high-frequency corrections at 4 "touch ticks" per vision tick. Forces change faster than frames arrive, so the architecture had to as well. 2. Open data. The largest tactile dataset ever released to our knowledge: a 50-hour (~5,500 episodes) high-quality, carefully synchronized robot play corpus, collected on SOTA tactile hand hardware with 22 degrees of freedom. Available today on HuggingFace! 3. Training recipe: T-Rex extends our prior work, EgoScale. Human egocentric videos for pretraining, a diverse dose of tactile robot play for mid-training. Our experiments show this bridges contact-free pretraining to contact-rich manipulation remarkably well. Pixels are cheap and everywhere, but they run out of steam at the moment of contact. Tactile will carry the last mile. The next scaling curve will be measured in hours of touch. T-Rex is a great collaboration between NVIDIA and Berkeley: 🧵

Jim Fan

182,032 Aufrufe • vor 1 Monat

Robots need to feel the world to operate in it. Most manipulation policies today are tactile-blind. They either cannot interpret high-frequency tactile signals or treat them as a static channel. And the field lacks enough touch-rich datasets to train tactile-reactive policies at scale. T-Rex was built to answer both. Dantong Niu and team, advised by Jim Fan, Fei-Fei Li, Jitendra MALIK, Pieter Abbeel trevordarrell have tested whether a robot policy can react to high-frequency tactile signals the way human hands do, without giving up the generalization power of modern VLAs. The result: 65% average success rate across 12 real-world tasks. +30 absolute points over the strongest baseline. In one year, tactile VLAs have gone from promising to outperforming non-tactile baselines like pi0.5 on dexterous tasks. #Robotics #SharpaWave #Sharpa #EmbodiedAI #DexterousManipulation #TactileSensing #RobotLearning Project link:

Robots need to feel the world to operate in it. Most manipulation policies today are tactile-blind. They either cannot interpret high-frequency tactile signals or treat them as a static channel. And the field lacks enough touch-rich datasets to train tactile-reactive policies at scale. T-Rex was built to answer both. Dantong Niu and team, advised by Jim Fan, Fei-Fei Li, Jitendra MALIK, Pieter Abbeel trevordarrell have tested whether a robot policy can react to high-frequency tactile signals the way human hands do, without giving up the generalization power of modern VLAs. The result: 65% average success rate across 12 real-world tasks. +30 absolute points over the strongest baseline. In one year, tactile VLAs have gone from promising to outperforming non-tactile baselines like pi0.5 on dexterous tasks. #Robotics #SharpaWave #Sharpa #EmbodiedAI #DexterousManipulation #TactileSensing #RobotLearning Project link:

Sharpa

12,957 Aufrufe • vor 3 Monaten

Who can say no to a humanoid robot that can actually feel a hug? 🤖 JQ Industries just built a fabric-based electronic skin for Unitree G1 — covering the torso, hands, and feet. This is not about appearance. It’s about giving robots real tactile sensing and pressure feedback. According to the company, the system includes tactile gloves, upper-body skin, and foot sensors. That means the robot can feel contact through its hands, detect pressure across its body, and use foot feedback to adjust balance. This is the missing layer in most humanoids. Vision alone can’t handle physical interaction. Robots need touch to grip safely, react to humans, and stay stable in real-world environments. JQ says its fiber sensors can detect forces as low as 0.01N, handle up to 3500kPa, and bend below 0.1mm radius. The system can also integrate with Isaac Sim, making tactile data usable for training and deployment. Unitree G1 is already built for manipulation and control — this adds the sensing layer it lacked. JQ says it’s now working with multiple humanoid companies, with some systems already shipping at scale. This is what happens when robots stop just seeing the world — and start physically feeling it.

Who can say no to a humanoid robot that can actually feel a hug? 🤖 JQ Industries just built a fabric-based electronic skin for Unitree G1 — covering the torso, hands, and feet. This is not about appearance. It’s about giving robots real tactile sensing and pressure feedback. According to the company, the system includes tactile gloves, upper-body skin, and foot sensors. That means the robot can feel contact through its hands, detect pressure across its body, and use foot feedback to adjust balance. This is the missing layer in most humanoids. Vision alone can’t handle physical interaction. Robots need touch to grip safely, react to humans, and stay stable in real-world environments. JQ says its fiber sensors can detect forces as low as 0.01N, handle up to 3500kPa, and bend below 0.1mm radius. The system can also integrate with Isaac Sim, making tactile data usable for training and deployment. Unitree G1 is already built for manipulation and control — this adds the sensing layer it lacked. JQ says it’s now working with multiple humanoid companies, with some systems already shipping at scale. This is what happens when robots stop just seeing the world — and start physically feeling it.

RoboHub🤖

10,952 Aufrufe • vor 6 Monaten

We are back. After one year of quiet building. Introducing GENE-26.5, our first robotic brain that takes a major step toward human-level capability. For years, robotics has struggled to learn from the world’s largest and valuable data source: Humans. Solving it means rethinking the whole stack from the ground up: - A robotics-native foundation model. - A 1:1 human-like robotic hand. - A noninvasive data collection glove for motion, force, and touch. - A simulator that turns weeks of experiments into minutes. GENE-26.5 is trained across language, vision, proprioception, tactile, and action. We designed a set of tasks to test how far we can go with this new paradigm. Fully autonomous, 1x speed, one model, same weights. (Enjoy with sound on) We are approaching the endgame for robotics. And this is just a beginning.

We are back. After one year of quiet building. Introducing GENE-26.5, our first robotic brain that takes a major step toward human-level capability. For years, robotics has struggled to learn from the world’s largest and valuable data source: Humans. Solving it means rethinking the whole stack from the ground up: - A robotics-native foundation model. - A 1:1 human-like robotic hand. - A noninvasive data collection glove for motion, force, and touch. - A simulator that turns weeks of experiments into minutes. GENE-26.5 is trained across language, vision, proprioception, tactile, and action. We designed a set of tasks to test how far we can go with this new paradigm. Fully autonomous, 1x speed, one model, same weights. (Enjoy with sound on) We are approaching the endgame for robotics. And this is just a beginning.

Genesis AI

2,746,374 Aufrufe • vor 5 Monaten

TARS has made some big announcements at ICRA 2026. DexHand - A dexterous hand with 1:1 replication of human metacarpal and finger structure, multi-axis thumb opposition - Quasi-direct-drive + backdrivable joints, 21 degrees of freedom - All 26 ASL letter signs reproduced at high precision -0.05mm fingertip tactile sensing - Data. Model. Hand. Fused into one embodied AI-native dexterous hand. OmniVTA with TacForeSight A visuo-tactile world model for general-use, paired with TacForeSight, a force-guided technology that not only allows robots to understand contact but also enables them to predict contact before it happens. - Semantic touch ("slippery," "rough," "soft," "hard") enabling texture-based material cognition. - Operates inside a real-time closed loop - Contact state prediction and reaction at the microsecond scale, faster than human AWE 3.0 - Claimed to be the world's leading general-purpose embodied foundation model for real-world work - A1 robot packing demo as the showcase: long-horizon, bimanual, with autonomous error recovery and end-to-end imitation learning - Set a Guinness World Record in March 2026 for "Most sub-millimeter wire harness assembled by a robot in one hour" - SenseHub data acquisition suite captures human experience data to train AWE 3.0

TARS has made some big announcements at ICRA 2026. DexHand - A dexterous hand with 1:1 replication of human metacarpal and finger structure, multi-axis thumb opposition - Quasi-direct-drive + backdrivable joints, 21 degrees of freedom - All 26 ASL letter signs reproduced at high precision -0.05mm fingertip tactile sensing - Data. Model. Hand. Fused into one embodied AI-native dexterous hand. OmniVTA with TacForeSight A visuo-tactile world model for general-use, paired with TacForeSight, a force-guided technology that not only allows robots to understand contact but also enables them to predict contact before it happens. - Semantic touch ("slippery," "rough," "soft," "hard") enabling texture-based material cognition. - Operates inside a real-time closed loop - Contact state prediction and reaction at the microsecond scale, faster than human AWE 3.0 - Claimed to be the world's leading general-purpose embodied foundation model for real-world work - A1 robot packing demo as the showcase: long-horizon, bimanual, with autonomous error recovery and end-to-end imitation learning - Set a Guinness World Record in March 2026 for "Most sub-millimeter wire harness assembled by a robot in one hour" - SenseHub data acquisition suite captures human experience data to train AWE 3.0

The Humanoid Hub

11,322 Aufrufe • vor 4 Monaten

Feeling what a robot feels is becoming real! 🫰 Fluid Reality just ran its first full end-to-end touch teleoperation system, and the demo shows why high-resolution haptics might be the missing piece in robot control. A 22 mm fingertip display with 32 independently actuated “bubbles” was mapped directly to tactile sensors on a robot hand. 🫧 Three fingers streamed real-time touch data robot → human, giving the operator actual contact feedback instead of waving their hands blind, like most teleop systems today. It’s still early, but this is a real step toward dexterous ops and high-quality data for training Physical AI systems. Companies involved: Samsung Electronics, Alt-Bionics, Inc. (the robot hand), MANUS™ (hand tracking), and Sensible Robotics (fingertip sensors) Touch isn’t a nice-to-have in robotics. It’s the difference between manipulation that works in the real world P.S. Guess how many touch receptors we humans, have in our hand! :) ~~ ♻ Join the weekly robotics newsletter, and never miss any news →

Feeling what a robot feels is becoming real! 🫰 Fluid Reality just ran its first full end-to-end touch teleoperation system, and the demo shows why high-resolution haptics might be the missing piece in robot control. A 22 mm fingertip display with 32 independently actuated “bubbles” was mapped directly to tactile sensors on a robot hand. 🫧 Three fingers streamed real-time touch data robot → human, giving the operator actual contact feedback instead of waving their hands blind, like most teleop systems today. It’s still early, but this is a real step toward dexterous ops and high-quality data for training Physical AI systems. Companies involved: Samsung Electronics, Alt-Bionics, Inc. (the robot hand), MANUS™ (hand tracking), and Sensible Robotics (fingertip sensors) Touch isn’t a nice-to-have in robotics. It’s the difference between manipulation that works in the real world P.S. Guess how many touch receptors we humans, have in our hand! :) ~~ ♻ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

46,927 Aufrufe • vor 10 Monaten

Holy…S😳 Xiaomi's New CyberOne is so human-like Although this update features a bionic hand, I was immediately drawn to it. Let's look at the changes in the hand： It can handle industrial precision tasks like turning screws, plus delicate operations such as pinching feathers and throwing balloons. Behind the performance: >Volume cut by 60%:now almost identical in size/shape to a real human hand >Big leap in degrees of freedom (+50% total, +83% active:22-27DOF) >Full-palm tactile sensors over 8200 mm² for precise grip even without vision >150,000+ grip cycles durability (61-hour test) And a major innovation:Smart bionic sweat gland cooling: evaporates water for ~10W active heat dissipation Using tactile gloves to capture real human data, they’re training smoother, human-like grasps with imitation and reinforcement learning. Elon has said that humanoid hands and true AI are the most difficult aspects of building humanoid robots. It seems that Xiaomi is also getting close.

Holy…S😳 Xiaomi's New CyberOne is so human-like Although this update features a bionic hand, I was immediately drawn to it. Let's look at the changes in the hand： It can handle industrial precision tasks like turning screws, plus delicate operations such as pinching feathers and throwing balloons. Behind the performance: >Volume cut by 60%:now almost identical in size/shape to a real human hand >Big leap in degrees of freedom (+50% total, +83% active:22-27DOF) >Full-palm tactile sensors over 8200 mm² for precise grip even without vision >150,000+ grip cycles durability (61-hour test) And a major innovation:Smart bionic sweat gland cooling: evaporates water for ~10W active heat dissipation Using tactile gloves to capture real human data, they’re training smoother, human-like grasps with imitation and reinforcement learning. Elon has said that humanoid hands and true AI are the most difficult aspects of building humanoid robots. It seems that Xiaomi is also getting close.

CyberRobo

111,430 Aufrufe • vor 6 Monaten

How many of the big ideas of the past 15 years of AI are downstream of hardware constraints? The big hardware story over that period is that logic has become way cheaper than data transfer. Stacking huge numbers of matrix multiplies was perfect for this hardware regime, because matrix multiplication is logic-intensive but requires less data transfer. And so we got matmul-heavy deep learning. It's interesting to think about what AI would look like in a world where these costs didn't diverge so much.

How many of the big ideas of the past 15 years of AI are downstream of hardware constraints? The big hardware story over that period is that logic has become way cheaper than data transfer. Stacking huge numbers of matrix multiplies was perfect for this hardware regime, because matrix multiplication is logic-intensive but requires less data transfer. And so we got matmul-heavy deep learning. It's interesting to think about what AI would look like in a world where these costs didn't diverge so much.

Dwarkesh Patel

61,147 Aufrufe • vor 3 Monaten

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data. 2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the motion of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro -> RoboCasa produces N (varying visuals) -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are building tools to enable everyone in the ecosystem to scale up with us. Links in thread:

Jim Fan

364,721 Aufrufe • vor 2 Jahren

Jason Ma reveals the first true scaling law in robotics: robots trained on 1 million hours of first-person human video predictably get better without ever seeing robot data "Yesterday we announced our new flagship robot foundation model called Dyna-2. It's very significant for the entire field for many different reasons. First of all, it's the first robot foundation model trained on at least one million hours of data. That in itself is a very challenging infrastructure challenge." "What we demonstrated is that even if the one million hours of data is entirely first-person video of humans doing manipulation tasks, we actually see scaling law for transferring to robot embodiments that the model has never seen before." "By training on more human data, we actually see predictable performance improvement on robots. That's a big deal because the biggest challenge facing robot foundation models is that we don't have enough data." "Unlike language models, there are just not enough robotics data out there, and it's very difficult to collect these robot data. Only by passively observing humans doing things can we scale this kind of data, and this is the first time we showed there is a lot of improvement that can transfer." Jason Ma Dyna Robotics

Jason Ma reveals the first true scaling law in robotics: robots trained on 1 million hours of first-person human video predictably get better without ever seeing robot data "Yesterday we announced our new flagship robot foundation model called Dyna-2. It's very significant for the entire field for many different reasons. First of all, it's the first robot foundation model trained on at least one million hours of data. That in itself is a very challenging infrastructure challenge." "What we demonstrated is that even if the one million hours of data is entirely first-person video of humans doing manipulation tasks, we actually see scaling law for transferring to robot embodiments that the model has never seen before." "By training on more human data, we actually see predictable performance improvement on robots. That's a big deal because the biggest challenge facing robot foundation models is that we don't have enough data." "Unlike language models, there are just not enough robotics data out there, and it's very difficult to collect these robot data. Only by passively observing humans doing things can we scale this kind of data, and this is the first time we showed there is a lot of improvement that can transfer." Jason Ma Dyna Robotics

MTS

17,889 Aufrufe • vor 2 Monaten

MEET BLUE: NVIDIA’S REAL-LIFE WALL-E Blue is an AI robot that moves, reacts, and talks like something straight out of a Pixar film. Blue runs on 2 NVIDIA computers and uses a physics engine for ultra-realistic tactile feedback, rigid and soft body simulation, all in real time. This isn’t a concept, this is how NVIDIA plans to train robots for the real world. And yes, it casually stole the show. Source: entrepreneurialogy

MEET BLUE: NVIDIA’S REAL-LIFE WALL-E Blue is an AI robot that moves, reacts, and talks like something straight out of a Pixar film. Blue runs on 2 NVIDIA computers and uses a physics engine for ultra-realistic tactile feedback, rigid and soft body simulation, all in real time. This isn’t a concept, this is how NVIDIA plans to train robots for the real world. And yes, it casually stole the show. Source: entrepreneurialogy

Mario Nawfal

38,388 Aufrufe • vor 1 Jahr

Sharpa Robotics just dropped a new hand video and the level keeps going up. This is the Sharpa Wave running WM Craftnet on a human scale fivefinger hand with 22 active DoF. The policy combines wrist depth, tactile sensing, proprioception and previous actions. The hand can rotate different objects in-hand, recover after external pushes and continue manipulating objects it was never trained on. The numbers are strong. 175/200 successful real-world rotation trials across 20 objects. A world-model prior trained on 9 objects was transferred to 49 new objects. Fall rate went from 6% to 0.3%. The Wave hardware itself has 22 actuators, up to 20 N fingertip force, 240×240 tactile sensing at up to 180 fps and 0.02 N pressure sensitivity. What caught my attention is the recovery behavior. The fingers keep changing contact points after the object slips or gets pushed instead of replaying the same finger motion. That is the kind of dexterity I want to see more of in robotic hands. According to Sharpa’s current specifications: • DTA tactile sensors on the fingers with a resolution of up to 240 × 240 • Pressure detection • Slip detection • Force change detection • Contact point localization • 6-axis force and torque measurement: Fx, Fy, Fz, Mx, My, Mz • Tactile sensing at up to 180 fps • 20 ms reported latency • Force detection range from 0 to 30 N • Maximum sensor load of 50 N • Sharpa also describes a miniature camera integrated into each fingertip for visuo-tactile sensing.

Sharpa Robotics just dropped a new hand video and the level keeps going up. This is the Sharpa Wave running WM Craftnet on a human scale fivefinger hand with 22 active DoF. The policy combines wrist depth, tactile sensing, proprioception and previous actions. The hand can rotate different objects in-hand, recover after external pushes and continue manipulating objects it was never trained on. The numbers are strong. 175/200 successful real-world rotation trials across 20 objects. A world-model prior trained on 9 objects was transferred to 49 new objects. Fall rate went from 6% to 0.3%. The Wave hardware itself has 22 actuators, up to 20 N fingertip force, 240×240 tactile sensing at up to 180 fps and 0.02 N pressure sensitivity. What caught my attention is the recovery behavior. The fingers keep changing contact points after the object slips or gets pushed instead of replaying the same finger motion. That is the kind of dexterity I want to see more of in robotic hands. According to Sharpa’s current specifications: • DTA tactile sensors on the fingers with a resolution of up to 240 × 240 • Pressure detection • Slip detection • Force change detection • Contact point localization • 6-axis force and torque measurement: Fx, Fy, Fz, Mx, My, Mz • Tactile sensing at up to 180 fps • 20 ms reported latency • Force detection range from 0 to 30 N • Maximum sensor load of 50 N • Sharpa also describes a miniature camera integrated into each fingertip for visuo-tactile sensing.

Techniahqrobot | humanoid robots

13,730 Aufrufe • vor 23 Tagen

How does an FDE map the processes to deploy AI in a real business? We use 3 sources. Source 1 is people. - Companies tell us: "this person's been at the firm for 20 years and they just handle it." - We hear this from Fortune 50 companies and from SMBs. - So in finance, you talk to the head of AP, AR, reconciliations, billing, banking, and FP&A. You make a "human API." Source 2 is the systems of record. - With real-time access to Salesforce for 3 to 4 weeks, you see what data comes in and how often people correct it. Source 3 is the documents in SharePoint, Drive, Notion, Slack, Teams, and Gmail. If you use the system data without the interviews, you miss half of the picture. The split is different for each company. A large company can have 10 years of Salesforce data. An SMB has most of it in people's heads. Choose your angle for each company before you build.

How does an FDE map the processes to deploy AI in a real business? We use 3 sources. Source 1 is people. - Companies tell us: "this person's been at the firm for 20 years and they just handle it." - We hear this from Fortune 50 companies and from SMBs. - So in finance, you talk to the head of AP, AR, reconciliations, billing, banking, and FP&A. You make a "human API." Source 2 is the systems of record. - With real-time access to Salesforce for 3 to 4 weeks, you see what data comes in and how often people correct it. Source 3 is the documents in SharePoint, Drive, Notion, Slack, Teams, and Gmail. If you use the system data without the interviews, you miss half of the picture. The split is different for each company. A large company can have 10 years of Salesforce data. An SMB has most of it in people's heads. Choose your angle for each company before you build.

The Startup Ideas Podcast (SIP) 🧃

35,097 Aufrufe • vor 10 Tagen