Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

X Square Robot has introduced TwinDEX: a co-designed pair of three-finger, 9-DoF dexterous hands, one wearable for data collection, one matched robot end effector for deployment. The wearable and the robot hand share the same kinematics, geometry, materials, look, and sensors, so human demos map straight to the robot... show more

The Humanoid Hub

115,739 subscribers

276,119 Aufrufe • vor 1 Tag •via X (Twitter)

Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

What if a dexterous robot could learn a 20+ step chemistry experiment without a single on-robot training demo? Meet TwinDEX: a pair of co-designed, three-finger, nine-DoF dexterous manipulation interface: one wearable for data collection, one for robot deployment. The twinned design shares identical kinematics, contact surfaces, visual appearance, and sensors across collection and deployment — keeping observations and actions aligned end to end. Trained from scratch on only a few hundred wearable demonstrations - with zero on-robot training or intervention data - TwinDEX completed a standardized chemistry experiment involving tool switches, fine force control, and bimanual coordination. Robot-free data showed comparable learning efficiency on the multi-task evaluation, TwinDEX delivered 5.3 times effective throughput than on-robot teleoperation. TwinDEX demonstrates that high-quality robot-free data can fully substitute for on-robot teleoperation data on challenging dexterous tasks — removing the dependency on real-robot hardware that has been the central bottleneck to scaling dexterous manipulation data. This was the proof-it phase. Now comes scale: what emerges at tens of thousands, or millions, of episodes? Watch the demo and read the technical blog: #TwinDEX #Robotics #EmbodiedAI #DexterousManipulation

What if a dexterous robot could learn a 20+ step chemistry experiment without a single on-robot training demo? Meet TwinDEX: a pair of co-designed, three-finger, nine-DoF dexterous manipulation interface: one wearable for data collection, one for robot deployment. The twinned design shares identical kinematics, contact surfaces, visual appearance, and sensors across collection and deployment — keeping observations and actions aligned end to end. Trained from scratch on only a few hundred wearable demonstrations - with zero on-robot training or intervention data - TwinDEX completed a standardized chemistry experiment involving tool switches, fine force control, and bimanual coordination. Robot-free data showed comparable learning efficiency on the multi-task evaluation, TwinDEX delivered 5.3 times effective throughput than on-robot teleoperation. TwinDEX demonstrates that high-quality robot-free data can fully substitute for on-robot teleoperation data on challenging dexterous tasks — removing the dependency on real-robot hardware that has been the central bottleneck to scaling dexterous manipulation data. This was the proof-it phase. Now comes scale: what emerges at tens of thousands, or millions, of episodes? Watch the demo and read the technical blog: #TwinDEX #Robotics #EmbodiedAI #DexterousManipulation

X Square Robot

1,336,300 Aufrufe • vor 2 Tagen

A robot performing a chemistry experiment might not sound that surprising. What’s interesting about this demo is how the robot was trained. Chinese embodied AI startup X Square Robot's TwinDEX uses a wearable three-finger end-effector to collect demonstrations from a human, rather than requiring the robot to be teleoperated during data collection. The resulting policy handles a 20+ step chemistry workflow: opening reagent bottles, scooping material, operating a pipette, transferring liquids, handling a glass rod, pouring, and coordinating both hands throughout the process. These are very different from simple pick-and-place tasks. There are continuous contacts, tool use, changing grasps, precise positioning, force control, and long-horizon state transitions. The reported experiments used robot-free demonstrations only, with zero on-robot teleoperation data for training. The bigger idea here is that if the data-collection hardware and deployment hardware are designed to be sufficiently consistent, robot-free demonstrations can still retain the information needed for real dexterous manipulation.

A robot performing a chemistry experiment might not sound that surprising. What’s interesting about this demo is how the robot was trained. Chinese embodied AI startup X Square Robot's TwinDEX uses a wearable three-finger end-effector to collect demonstrations from a human, rather than requiring the robot to be teleoperated during data collection. The resulting policy handles a 20+ step chemistry workflow: opening reagent bottles, scooping material, operating a pipette, transferring liquids, handling a glass rod, pouring, and coordinating both hands throughout the process. These are very different from simple pick-and-place tasks. There are continuous contacts, tool use, changing grasps, precise positioning, force control, and long-horizon state transitions. The reported experiments used robot-free demonstrations only, with zero on-robot teleoperation data for training. The bigger idea here is that if the data-collection hardware and deployment hardware are designed to be sufficiently consistent, robot-free demonstrations can still retain the information needed for real dexterous manipulation.

Sharon Riley

18,036 Aufrufe • vor 2 Tagen

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

42,864 Aufrufe • vor 7 Monaten

Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced

Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced

Chen Wang

235,224 Aufrufe • vor 2 Jahren

The problem with humanoid teleoperation is that it is expensive and difficult to scale Enter NVIDIA's EgoScale: - A VLA model pretrained on thousands hours of egocentric human videos. - Mid-trained via 50 hours of human + 4 hours of robot "play" data for human-robot alignment. - Fine-tuned with very few examples of task-specific robot teleoperation (100 or fewer per task). - Successfully transfers across 5-finger (Sharpa) and 3-finger (Unitree G1) robot hands. - Performance scales predictably as data increases.

The problem with humanoid teleoperation is that it is expensive and difficult to scale Enter NVIDIA's EgoScale: - A VLA model pretrained on thousands hours of egocentric human videos. - Mid-trained via 50 hours of human + 4 hours of robot "play" data for human-robot alignment. - Fine-tuned with very few examples of task-specific robot teleoperation (100 or fewer per task). - Successfully transfers across 5-finger (Sharpa) and 3-finger (Unitree G1) robot hands. - Performance scales predictably as data increases.

The Humanoid Hub

44,441 Aufrufe • vor 6 Monaten

Mimic Robotics released a full-stack dexterity platform: the mimic hand M1 + umimic (U1) wearable. They keep human hand morphology consistent across the entire data-model-dexterous hand-manipulation pipeline--from video pre-training to wearable demos and robot deployment. This avoids cross-embodiment gaps and captures the essence of upper-limb intelligence. M1: Tendon-driven (15 active DoF), highly backdrivable ( 25kg grasp, industrial-grade, Swiss-made. U1 lets humans teach the robot naturally with their own hands--1:1 kinematics and sensing match, no latency or retargeting, enabling higher-quality and more scalable data. Looking forward to its performance at the customer site.

Mimic Robotics released a full-stack dexterity platform: the mimic hand M1 + umimic (U1) wearable. They keep human hand morphology consistent across the entire data-model-dexterous hand-manipulation pipeline--from video pre-training to wearable demos and robot deployment. This avoids cross-embodiment gaps and captures the essence of upper-limb intelligence. M1: Tendon-driven (15 active DoF), highly backdrivable ( 25kg grasp, industrial-grade, Swiss-made. U1 lets humans teach the robot naturally with their own hands--1:1 kinematics and sensing match, no latency or retargeting, enabling higher-quality and more scalable data. Looking forward to its performance at the customer site.

CyberRobo

26,160 Aufrufe • vor 1 Monat

One approach to collecting data for contact-rich manipulation is handheld grippers. These devices let the operator feel contact with objects and the environment directly through the mechanism. To make sure the data collected transfers to the robot, we've co-designed handheld and robot grippers: same linkage mechanisms, same degrees of freedom, same force transmission. The Koala platform lets the human feel through the linkages what the robot will feel through its actuators. Watch to learn more about our gripper designs, data collection approach, and policy training:

One approach to collecting data for contact-rich manipulation is handheld grippers. These devices let the operator feel contact with objects and the environment directly through the mechanism. To make sure the data collected transfers to the robot, we've co-designed handheld and robot grippers: same linkage mechanisms, same degrees of freedom, same force transmission. The Koala platform lets the human feel through the linkages what the robot will feel through its actuators. Watch to learn more about our gripper designs, data collection approach, and policy training:

RAI Institute

23,396 Aufrufe • vor 3 Monaten

This year is truly the dawn of the “robot data oil” rush--spawning all kinds of pick-and-shovel tools to dig it up. X Square Robot, which offers in-home humanoid cleaning services, has just launched a full UMI/Egocentric data collection system: QUANXTA Zero. Real robots delivering actual on-site services is one powerful data collection method. Meanwhile, bodiless (no physical robot) UMI/Ego-centric( human-centric)data pipelines are also surging in popularity. Well,everyone’s racing to distill the physical world and extract the data that truly matters for advancing robot intelligence.

This year is truly the dawn of the “robot data oil” rush--spawning all kinds of pick-and-shovel tools to dig it up. X Square Robot, which offers in-home humanoid cleaning services, has just launched a full UMI/Egocentric data collection system: QUANXTA Zero. Real robots delivering actual on-site services is one powerful data collection method. Meanwhile, bodiless (no physical robot) UMI/Ego-centric( human-centric)data pipelines are also surging in popularity. Well,everyone’s racing to distill the physical world and extract the data that truly matters for advancing robot intelligence.

CyberRobo

21,950 Aufrufe • vor 2 Monaten

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 Aufrufe • vor 1 Jahr

Happy to share what I’ve been working on since joining Genesis! GENE-26.5 is a one-of-a-kind, robotics-native multimodal foundation model that learns from diverse, in-the-wild data across modalities and outputs actions enabling a 54-DoF robot system to perform the most dexterous, long-horizon manipulation tasks to date—approaching human-level capability. This is the result of innovations across the full stack—data collection and processing, robot systems, model architecture, training strategies, and scalable evaluation infrastructure.

Happy to share what I’ve been working on since joining Genesis! GENE-26.5 is a one-of-a-kind, robotics-native multimodal foundation model that learns from diverse, in-the-wild data across modalities and outputs actions enabling a 54-DoF robot system to perform the most dexterous, long-horizon manipulation tasks to date—approaching human-level capability. This is the result of innovations across the full stack—data collection and processing, robot systems, model architecture, training strategies, and scalable evaluation infrastructure.

Zu Wang

20,105 Aufrufe • vor 4 Monaten

Not every robot needs a humanlike head. 🇰🇷🤖 South Korea’s XYZ unveiled DEUX, a dual-arm semi-humanoid built for everyday spaces. DEUX pairs two 7-DoF arms with two 7-DoF three-finger hands, a swerve-drive base, and an 11 kg dual-arm payload. XYZ already builds robot baristas and indoor delivery robots. Its Physical AI pipeline runs from Glove X data collection to model training, robot deployment, and real-world validation. Keep the arms and mobility. Skip the anatomy you don’t need.

Not every robot needs a humanlike head. 🇰🇷🤖 South Korea’s XYZ unveiled DEUX, a dual-arm semi-humanoid built for everyday spaces. DEUX pairs two 7-DoF arms with two 7-DoF three-finger hands, a swerve-drive base, and an 11 kg dual-arm payload. XYZ already builds robot baristas and indoor delivery robots. Its Physical AI pipeline runs from Glove X data collection to model training, robot deployment, and real-world validation. Keep the arms and mobility. Skip the anatomy you don’t need.

RoboHub🤖

40,268 Aufrufe • vor 1 Monat

We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop. Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate. Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution. Our recipe is called "EgoScale": - Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks. - Most surprising result: a *single* teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency. - Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone. The scalable path to robot dexterity was never more robots. It was always us. Deep dives in thread:

We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop. Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate. Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution. Our recipe is called "EgoScale": - Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks. - Most surprising result: a single teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency. - Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone. The scalable path to robot dexterity was never more robots. It was always us. Deep dives in thread:

Jim Fan

299,688 Aufrufe • vor 6 Monaten

Presenting DemoDiffusion: An extremely simple approach enabling a pre-trained 'generalist' diffusion policy to follow a human-demonstration for a novel task during inference One-shot human imitation *without* requiring any paired human-robot data or online RL 🙂 1/n

Presenting DemoDiffusion: An extremely simple approach enabling a pre-trained 'generalist' diffusion policy to follow a human-demonstration for a novel task during inference One-shot human imitation without requiring any paired human-robot data or online RL 🙂 1/n

Homanga Bharadhwaj

32,919 Aufrufe • vor 1 Jahr

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 Aufrufe • vor 1 Jahr

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

NVIDIA Robotics

22,958 Aufrufe • vor 1 Monat

Mecka is excited to power EgoVerse: a growing ecosystem for robot learning from egocentric human data Proven across 4 leading research labs, EgoVerse data consistently boosts robot performance We release the tools for anyone to collect data, inference/train, and contribute! 🧵

Mecka is excited to power EgoVerse: a growing ecosystem for robot learning from egocentric human data Proven across 4 leading research labs, EgoVerse data consistently boosts robot performance We release the tools for anyone to collect data, inference/train, and contribute! 🧵

Josh

103,539 Aufrufe • vor 5 Monaten

Open-source robot arm meets hand tracking [📍GitHub below] It is designed with an industrial mindset but built as a 3D-printed desktop system. PAROL6 paired with a LEAP Motion controller is a nice example of how accessible robot teleoperation has become. • Hand motion is streamed to the robot at 100 Hz via UDP • A pneumatic gripper is controlled by simple fist open and close gestures • The entire robot stack is open source, from mechanics to control software Combine that with low-latency hand tracking and you get a very practical platform for learning manipulation, teleoperation, and human-robot interfaces. This kind of setup is great for experimentation, teleop, data collection, and teaching robots by demonstration All without proprietary hardware or locked software. Credit to SourceRobotics 📍Code: —— Weekly robotics and AI insights. Subscribe free:

Open-source robot arm meets hand tracking [📍GitHub below] It is designed with an industrial mindset but built as a 3D-printed desktop system. PAROL6 paired with a LEAP Motion controller is a nice example of how accessible robot teleoperation has become. • Hand motion is streamed to the robot at 100 Hz via UDP • A pneumatic gripper is controlled by simple fist open and close gestures • The entire robot stack is open source, from mechanics to control software Combine that with low-latency hand tracking and you get a very practical platform for learning manipulation, teleoperation, and human-robot interfaces. This kind of setup is great for experimentation, teleop, data collection, and teaching robots by demonstration All without proprietary hardware or locked software. Credit to SourceRobotics 📍Code: —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

32,025 Aufrufe • vor 7 Monaten

A modular data glove built for robot learning. ENSURING’s Magic Glove uses quick-release sensor modules, combining 15 motion-tracking units, 1,000+ tactile sensing points, and real-time 6-axis force data. The modular design is the interesting part here: sensors can be swapped or reconfigured depending on the task, making it a more flexible interface for teleoperation, dexterous manipulation, and embodied AI data collection.

A modular data glove built for robot learning. ENSURING’s Magic Glove uses quick-release sensor modules, combining 15 motion-tracking units, 1,000+ tactile sensing points, and real-time 6-axis force data. The modular design is the interesting part here: sensors can be swapped or reconfigured depending on the task, making it a more flexible interface for teleoperation, dexterous manipulation, and embodied AI data collection.

Eren Chen

33,006 Aufrufe • vor 18 Tagen

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵

Mahi Shafiullah 🏠🤖

95,070 Aufrufe • vor 2 Monaten

Feeling what a robot feels is becoming real! 🫰 Fluid Reality just ran its first full end-to-end touch teleoperation system, and the demo shows why high-resolution haptics might be the missing piece in robot control. A 22 mm fingertip display with 32 independently actuated “bubbles” was mapped directly to tactile sensors on a robot hand. 🫧 Three fingers streamed real-time touch data robot → human, giving the operator actual contact feedback instead of waving their hands blind, like most teleop systems today. It’s still early, but this is a real step toward dexterous ops and high-quality data for training Physical AI systems. Companies involved: Samsung Electronics, Alt-Bionics, Inc. (the robot hand), MANUS™ (hand tracking), and Sensible Robotics (fingertip sensors) Touch isn’t a nice-to-have in robotics. It’s the difference between manipulation that works in the real world P.S. Guess how many touch receptors we humans, have in our hand! :) ~~ ♻ Join the weekly robotics newsletter, and never miss any news →

Feeling what a robot feels is becoming real! 🫰 Fluid Reality just ran its first full end-to-end touch teleoperation system, and the demo shows why high-resolution haptics might be the missing piece in robot control. A 22 mm fingertip display with 32 independently actuated “bubbles” was mapped directly to tactile sensors on a robot hand. 🫧 Three fingers streamed real-time touch data robot → human, giving the operator actual contact feedback instead of waving their hands blind, like most teleop systems today. It’s still early, but this is a real step toward dexterous ops and high-quality data for training Physical AI systems. Companies involved: Samsung Electronics, Alt-Bionics, Inc. (the robot hand), MANUS™ (hand tracking), and Sensible Robotics (fingertip sensors) Touch isn’t a nice-to-have in robotics. It’s the difference between manipulation that works in the real world P.S. Guess how many touch receptors we humans, have in our hand! :) ~~ ♻ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

46,927 Aufrufe • vor 9 Monaten