Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

The Koala gripper is a co-designed system that pairs a handheld data collection device and a robotic gripper, allowing a single operator to easily record task demonstrations. Engineered with three controllable degrees of freedom and force-optimized triggers, the device enables complex, human-like object interactions. Training on a dataset of... show more

RAI Institute

8,133 subscribers

18,978 Aufrufe • vor 1 Monat •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

2 Kommentare

Profilbild von Clara Lafever Jane

Clara Lafever Janevor 1 Monat

No more demos please Let's do more deployments This shit is getting old 😒

Profilbild von Soph w

Soph wvor 1 Monat

Facts!

Ähnliche Videos

Researchers at RAI Institute developed the Koala gripper to help robots handle tools and objects more naturally. The system includes a handheld version for humans and a motorized version for robots, both using the same design and sensors. By learning from human demonstrations recorded with cameras and force sensors, robots can repeat the same tasks using the Koala gripper.

Researchers at RAI Institute developed the Koala gripper to help robots handle tools and objects more naturally. The system includes a handheld version for humans and a motorized version for robots, both using the same design and sensors. By learning from human demonstrations recorded with cameras and force sensors, robots can repeat the same tasks using the Koala gripper.

Space and Technology

22,169 Aufrufe • vor 3 Monaten

One approach to collecting data for contact-rich manipulation is handheld grippers. These devices let the operator feel contact with objects and the environment directly through the mechanism. To make sure the data collected transfers to the robot, we've co-designed handheld and robot grippers: same linkage mechanisms, same degrees of freedom, same force transmission. The Koala platform lets the human feel through the linkages what the robot will feel through its actuators. Watch to learn more about our gripper designs, data collection approach, and policy training:

One approach to collecting data for contact-rich manipulation is handheld grippers. These devices let the operator feel contact with objects and the environment directly through the mechanism. To make sure the data collected transfers to the robot, we've co-designed handheld and robot grippers: same linkage mechanisms, same degrees of freedom, same force transmission. The Koala platform lets the human feel through the linkages what the robot will feel through its actuators. Watch to learn more about our gripper designs, data collection approach, and policy training:

RAI Institute

23,538 Aufrufe • vor 5 Monaten

A gripper with two thumbs! 🐨 Handheld data collection is how a lot of manipulation datasets get built now. Someone picks up a gripper-shaped device, does the task by hand, and the recording becomes training data. Voila! 🤌🏼 The catch is that these devices are usually shaped to match whatever robot gripper already exists. The human ends up working around the robot's morphology, which costs both ergonomics and the quality of the demonstration being recorded. The RAI Institute team designed The Koala Gripper. It includes two devices with full parity: a capture device a person holds, and a robotic device that executes the learned policy. Anything you can do with one, the other reproduces. The shared feature set gets updated as the design iterates, so neither side quietly compromises the other. The morphology is where it gets interesting. Two independently controlled underactuated fingers are opposed by a single pivoting "dual thumb", which is where the koala name comes from. → Each finger is a 9-bar linkage with one actuated and one underactuated degree of freedom, with the spring sitting proximal to the phalanges to drive preshaping → Fingernails on the distal link for precise pinches, contoured pads for secure wrap grasps → The handheld trigger traces a natural fingertip arc instead of a straight line, lengthening the stroke and giving the operator better mechanical advantage → On the robot side, a frameless motor with an integrated high-pitch ballscrew keeps reflected inertia low, so the fingers stay backdrivable at an effective mass of tens of grams The dual thumb is what buys tool stabilising, singulating one object out of a pile, and tabletop pinches. Congrats to Zubin Kremer Guha and the RAI Institute team behind it 👏🏼 🔗 Project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

A gripper with two thumbs! 🐨 Handheld data collection is how a lot of manipulation datasets get built now. Someone picks up a gripper-shaped device, does the task by hand, and the recording becomes training data. Voila! 🤌🏼 The catch is that these devices are usually shaped to match whatever robot gripper already exists. The human ends up working around the robot's morphology, which costs both ergonomics and the quality of the demonstration being recorded. The RAI Institute team designed The Koala Gripper. It includes two devices with full parity: a capture device a person holds, and a robotic device that executes the learned policy. Anything you can do with one, the other reproduces. The shared feature set gets updated as the design iterates, so neither side quietly compromises the other. The morphology is where it gets interesting. Two independently controlled underactuated fingers are opposed by a single pivoting "dual thumb", which is where the koala name comes from. → Each finger is a 9-bar linkage with one actuated and one underactuated degree of freedom, with the spring sitting proximal to the phalanges to drive preshaping → Fingernails on the distal link for precise pinches, contoured pads for secure wrap grasps → The handheld trigger traces a natural fingertip arc instead of a straight line, lengthening the stroke and giving the operator better mechanical advantage → On the robot side, a frameless motor with an integrated high-pitch ballscrew keeps reflected inertia low, so the fingers stay backdrivable at an effective mass of tens of grams The dual thumb is what buys tool stabilising, singulating one object out of a pile, and tabletop pinches. Congrats to Zubin Kremer Guha and the RAI Institute team behind it 👏🏼 🔗 Project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

19,233 Aufrufe • vor 1 Monat

.Jason Ma on how Dyna-2 learned to untwist a real water bottle cap out of the box from just 13 minutes of dexterous hand data because the robot hand looks like a human's: "We got a new pair of dexterous hand. Most of our robots in the office have a gripper-like hand, so the gap between that versus human is actually quite large." "We thought, if we have a robot that has hands that look like human, wouldn't the transfer be greater? Right before the release, they were able to collect just 13 minutes of data." "They collected only 13 minutes of data and then post-trained the best Dyna model that trained on one million hours of data. The model is just able to, out of the box, start opening bottle caps on real water bottles." "This is very surprising because typically post-training a robot would take at least several hours of data. For a dexterous hand that's much more complex than a gripper, it would actually be much harder to train. But the experimental results we saw was the opposite. Perhaps because the hand looks more like the human, the transfer was easier." Dyna Robotics

.Jason Ma on how Dyna-2 learned to untwist a real water bottle cap out of the box from just 13 minutes of dexterous hand data because the robot hand looks like a human's: "We got a new pair of dexterous hand. Most of our robots in the office have a gripper-like hand, so the gap between that versus human is actually quite large." "We thought, if we have a robot that has hands that look like human, wouldn't the transfer be greater? Right before the release, they were able to collect just 13 minutes of data." "They collected only 13 minutes of data and then post-trained the best Dyna model that trained on one million hours of data. The model is just able to, out of the box, start opening bottle caps on real water bottles." "This is very surprising because typically post-training a robot would take at least several hours of data. For a dexterous hand that's much more complex than a gripper, it would actually be much harder to train. But the experimental results we saw was the opposite. Perhaps because the hand looks more like the human, the transfer was easier." Dyna Robotics

MTS

13,743 Aufrufe • vor 1 Monat

🎙️ Excited to introduce one of my favorite projects from the past year: TeleDexter, from the BIGAI dexterity team. It’s a stable, human-level dexterous teleoperation system and a suite of autonomous policies trained with it. Pen spinning, complex in-hand reorientation, and long-horizon tool use—once seen as the holy grail of manipulation—are now unlocked. 🧵👇 The hardware is already here; we have some incredible high-DoF robotic hands. The bottleneck? The controller. Most current systems are stuck in "quasi-static" grasping mode. Meanwhile, dynamic in-hand dexterity has remained severely limited. 🧠 To unlock the massive capabilities of human-like hands, we need to build an excellent "cerebellum" for dexterous hands. TeleDexter solves this with a novel co-tracking approach: it simultaneously tracks both human hand kinematics and object states, beautifully bridging the gap between human intent and robotic control. In order to train a better co-tracking policy that works robustly in the real world, we designed : (1) a hybrid reward design that combines consecutive goal reaching and dense tracking, (2) an action masking strategy during training that enhances sim2real performance, (3) a dexterous curriculum for learning the long-horizon interactions. Each design is inspired by numerous trials and countless real-world experiments. We’ve synthesized all the system details, engineering challenges, and core insights into our latest post. If you're interested in the future of dexterous manipulation, grab a coffee and check it out (9-min read): If you have more time, check out the paper:

🎙️ Excited to introduce one of my favorite projects from the past year: TeleDexter, from the BIGAI dexterity team. It’s a stable, human-level dexterous teleoperation system and a suite of autonomous policies trained with it. Pen spinning, complex in-hand reorientation, and long-horizon tool use—once seen as the holy grail of manipulation—are now unlocked. 🧵👇 The hardware is already here; we have some incredible high-DoF robotic hands. The bottleneck? The controller. Most current systems are stuck in "quasi-static" grasping mode. Meanwhile, dynamic in-hand dexterity has remained severely limited. 🧠 To unlock the massive capabilities of human-like hands, we need to build an excellent "cerebellum" for dexterous hands. TeleDexter solves this with a novel co-tracking approach: it simultaneously tracks both human hand kinematics and object states, beautifully bridging the gap between human intent and robotic control. In order to train a better co-tracking policy that works robustly in the real world, we designed : (1) a hybrid reward design that combines consecutive goal reaching and dense tracking, (2) an action masking strategy during training that enhances sim2real performance, (3) a dexterous curriculum for learning the long-horizon interactions. Each design is inspired by numerous trials and countless real-world experiments. We’ve synthesized all the system details, engineering challenges, and core insights into our latest post. If you're interested in the future of dexterous manipulation, grab a coffee and check it out (9-min read): If you have more time, check out the paper:

Siyuan Huang

12,746 Aufrufe • vor 2 Monaten

Egocentric Videos for Robot Training 🧵 Researchers at NYU and UC Berkeley published research where they developed a system called EgoZero that trains robots using human demonstration videos recorded with smart glasses. 👓 The system converts first-person human actions into 3D point-based state-action representations. The policies executed on a gripper-equipped robot, achieved a 70% zero-shot success rate across seven manipulation tasks, with only 20 minutes of human data per task. EgoZero stands as one of the solid empirical proofs that egocentric smart-glass video collected from everyday human behavior can serve as powerful, scalable training data for real robot learning. Vincent Liu Ademi Adeniji

Egocentric Videos for Robot Training 🧵 Researchers at NYU and UC Berkeley published research where they developed a system called EgoZero that trains robots using human demonstration videos recorded with smart glasses. 👓 The system converts first-person human actions into 3D point-based state-action representations. The policies executed on a gripper-equipped robot, achieved a 70% zero-shot success rate across seven manipulation tasks, with only 20 minutes of human data per task. EgoZero stands as one of the solid empirical proofs that egocentric smart-glass video collected from everyday human behavior can serve as powerful, scalable training data for real robot learning. Vincent Liu Ademi Adeniji

VaderResearch

23,055 Aufrufe • vor 1 Jahr

A lot of robot learning pipelines treat embodiment mismatch as a software problem. Collect data with one device, retarget the motion to another robot, and try to deal with the differences later. TwinDEX flips that idea around. This system is from a Chinese robotics company called X Square. Instead of starting with a generic data-collection device, the team co-designs a wearable three-finger end-effector and a robotic hand as two sides of the same learning system. They align the things that actually matter during manipulation: joint structure, link proportions, contact geometry, surface properties, visual appearance, sensing, and timing. That matters because dexterous manipulation is much more than matching hand poses. A few millimeters of contact error or a different force relationship can completely change how a task is executed. In the collection evaluation, TwinDEX delivered up to 5.3 times the effective throughput of on-robot teleoperation. It’s a useful reminder that scaling robot learning may not only be about bigger models and more data. Sometimes the hardware collecting the data is part of the data problem itself.

A lot of robot learning pipelines treat embodiment mismatch as a software problem. Collect data with one device, retarget the motion to another robot, and try to deal with the differences later. TwinDEX flips that idea around. This system is from a Chinese robotics company called X Square. Instead of starting with a generic data-collection device, the team co-designs a wearable three-finger end-effector and a robotic hand as two sides of the same learning system. They align the things that actually matter during manipulation: joint structure, link proportions, contact geometry, surface properties, visual appearance, sensing, and timing. That matters because dexterous manipulation is much more than matching hand poses. A few millimeters of contact error or a different force relationship can completely change how a task is executed. In the collection evaluation, TwinDEX delivered up to 5.3 times the effective throughput of on-robot teleoperation. It’s a useful reminder that scaling robot learning may not only be about bigger models and more data. Sometimes the hardware collecting the data is part of the data problem itself.

Alamin

48,112 Aufrufe • vor 1 Monat

Kinetix AI has teased KAI, its new humanoid robot -engineered with 36 degrees of freedom, more than any previous model, a hybrid dexterous hand, and 18,000 sensors embedded across a soft, flexible body-positioning it as the most human-like robotic system to date. Getting closer to real humanoid robots each day.

Kinetix AI has teased KAI, its new humanoid robot -engineered with 36 degrees of freedom, more than any previous model, a hybrid dexterous hand, and 18,000 sensors embedded across a soft, flexible body-positioning it as the most human-like robotic system to date. Getting closer to real humanoid robots each day.

Chubby♨️

22,473 Aufrufe • vor 5 Monaten

This is Clone Robotics realistic humanoid robotic arm, designed to closely mimic the movement and anatomy of a human arm. It is powered by electricity and a water-based hydraulic system using a compact pump about the size of a human heart. Despite weighing only 1 kilogram, the robotic arm can lift a 7-kilogram dumbbell.

This is Clone Robotics realistic humanoid robotic arm, designed to closely mimic the movement and anatomy of a human arm. It is powered by electricity and a water-based hydraulic system using a compact pump about the size of a human heart. Despite weighing only 1 kilogram, the robotic arm can lift a 7-kilogram dumbbell.

Space and Technology

58,940 Aufrufe • vor 3 Monaten

The Clone hand is a compact, biomimetic robotic hand designed to mimic human anatomy and perform a diverse array of manipulation tasks. Credit: Clone Robotics

The Clone hand is a compact, biomimetic robotic hand designed to mimic human anatomy and perform a diverse array of manipulation tasks. Credit: Clone Robotics

Tech Burrito

14,129 Aufrufe • vor 1 Jahr

NVIDIA just announced EgoScale 🤖🧠 NVIDIA Research has uncovered a log-linear scaling law for robot dexterity by pretraining VLA models on over 20,000 hours of egocentric human video This massive dataset is 20 times larger than previous efforts and proves that robot intelligence follows a predictable path: the more human data, the lower the loss The secret is a simple recipe combining large-scale human pretraining with a small amount of aligned human-robot mid-training to bridge the gap In testing, this method boosted the average success rate by 54% on a 22-DoF robotic hand compared to policies built without pretraining EgoScale also enables one-shot task adaptation and works across different hardware, suggesting that human motion is a universal motor prior for robots Website: Paper: Source: NVIDIA Research #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #NVIDIA #EgoScale #GR00T

NVIDIA just announced EgoScale 🤖🧠 NVIDIA Research has uncovered a log-linear scaling law for robot dexterity by pretraining VLA models on over 20,000 hours of egocentric human video This massive dataset is 20 times larger than previous efforts and proves that robot intelligence follows a predictable path: the more human data, the lower the loss The secret is a simple recipe combining large-scale human pretraining with a small amount of aligned human-robot mid-training to bridge the gap In testing, this method boosted the average success rate by 54% on a 22-DoF robotic hand compared to policies built without pretraining EgoScale also enables one-shot task adaptation and works across different hardware, suggesting that human motion is a universal motor prior for robots Website: Paper: Source: NVIDIA Research #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #NVIDIA #EgoScale #GR00T

RoboHub🤖

43,752 Aufrufe • vor 7 Monaten

Learning dexterous policies from human videos is challenging due to differences between human and robot hands. We present HuDOR, a method that learns dexterous policies within the robot's physical constraints using just one human video and an hour of online interactions! [1/n]

Learning dexterous policies from human videos is challenging due to differences between human and robot hands. We present HuDOR, a method that learns dexterous policies within the robot's physical constraints using just one human video and an hour of online interactions! [1/n]

Irmak Guzey

65,141 Aufrufe • vor 1 Jahr

X Square Robot has introduced TwinDEX: a co-designed pair of three-finger, 9-DoF dexterous hands, one wearable for data collection, one matched robot end effector for deployment. The wearable and the robot hand share the same kinematics, geometry, materials, look, and sensors, so human demos map straight to the robot without retargeting. Human data collection improves throughput 5.3x compared to teleoperation, and a policy can be trained with a few hundred episodes without any robot data.

X Square Robot has introduced TwinDEX: a co-designed pair of three-finger, 9-DoF dexterous hands, one wearable for data collection, one matched robot end effector for deployment. The wearable and the robot hand share the same kinematics, geometry, materials, look, and sensors, so human demos map straight to the robot without retargeting. Human data collection improves throughput 5.3x compared to teleoperation, and a policy can be trained with a few hundred episodes without any robot data.

The Humanoid Hub

277,022 Aufrufe • vor 1 Monat

What if a dexterous robot could learn a 20+ step chemistry experiment without a single on-robot training demo? Meet TwinDEX: a pair of co-designed, three-finger, nine-DoF dexterous manipulation interface: one wearable for data collection, one for robot deployment. The twinned design shares identical kinematics, contact surfaces, visual appearance, and sensors across collection and deployment — keeping observations and actions aligned end to end. Trained from scratch on only a few hundred wearable demonstrations - with zero on-robot training or intervention data - TwinDEX completed a standardized chemistry experiment involving tool switches, fine force control, and bimanual coordination. Robot-free data showed comparable learning efficiency on the multi-task evaluation, TwinDEX delivered 5.3 times effective throughput than on-robot teleoperation. TwinDEX demonstrates that high-quality robot-free data can fully substitute for on-robot teleoperation data on challenging dexterous tasks — removing the dependency on real-robot hardware that has been the central bottleneck to scaling dexterous manipulation data. This was the proof-it phase. Now comes scale: what emerges at tens of thousands, or millions, of episodes? Watch the demo and read the technical blog: #TwinDEX #Robotics #EmbodiedAI #DexterousManipulation

What if a dexterous robot could learn a 20+ step chemistry experiment without a single on-robot training demo? Meet TwinDEX: a pair of co-designed, three-finger, nine-DoF dexterous manipulation interface: one wearable for data collection, one for robot deployment. The twinned design shares identical kinematics, contact surfaces, visual appearance, and sensors across collection and deployment — keeping observations and actions aligned end to end. Trained from scratch on only a few hundred wearable demonstrations - with zero on-robot training or intervention data - TwinDEX completed a standardized chemistry experiment involving tool switches, fine force control, and bimanual coordination. Robot-free data showed comparable learning efficiency on the multi-task evaluation, TwinDEX delivered 5.3 times effective throughput than on-robot teleoperation. TwinDEX demonstrates that high-quality robot-free data can fully substitute for on-robot teleoperation data on challenging dexterous tasks — removing the dependency on real-robot hardware that has been the central bottleneck to scaling dexterous manipulation data. This was the proof-it phase. Now comes scale: what emerges at tens of thousands, or millions, of episodes? Watch the demo and read the technical blog: #TwinDEX #Robotics #EmbodiedAI #DexterousManipulation

X Square Robot

1,346,470 Aufrufe • vor 1 Monat

Will Cogley designed a highly bionic robotic head for a humanoid robot called the Stewart Platform Neck, demonstrating the ultimate pursuit of fine facial expressions and head posture control, which is inspiring for more realistic human-computer interaction in the future. The Stewart Platform consists of six linear actuators connecting two plates to achieve six degrees of freedom, simulating the complex structure of human neck muscles. It is controlled by Python programming and inverse kinematics algorithms, and supplemented by a 3D printed flexible spine to provide support and flexibility.

Will Cogley designed a highly bionic robotic head for a humanoid robot called the Stewart Platform Neck, demonstrating the ultimate pursuit of fine facial expressions and head posture control, which is inspiring for more realistic human-computer interaction in the future. The Stewart Platform consists of six linear actuators connecting two plates to achieve six degrees of freedom, simulating the complex structure of human neck muscles. It is controlled by Python programming and inverse kinematics algorithms, and supplemented by a 3D printed flexible spine to provide support and flexibility.

CyberRobo

32,437 Aufrufe • vor 1 Jahr

🔥 #ICRA2026 Best Paper Finalist The era of "robot VLA = single-arm gripper" is ending. Introducing Dexora — the first open-source Vision-Language-Action system for dual-arm, dual-hand, 36-DoF dexterous manipulation. 🦾 Dual Arms 🖐️ Dual Hands 🎯 36 DoF Control 🌍 Open Source Trained on: • 100K simulated trajectories • 10K real-world demonstrations Dexora achieves: ✓ 90%+ success on basic manipulation ✓ Strong dexterous manipulation performance ✓ Cross-embodiment generalization Our key hypothesis: Train on the hardest embodiment. Transfer to simpler robots later. Instead of scaling up gripper policies, we train directly in the most expressive action space and project downward to simpler embodiments. This may be a practical path toward universal robot controllers. 🎥 Demos: 📄 Paper:

🔥 #ICRA2026 Best Paper Finalist The era of "robot VLA = single-arm gripper" is ending. Introducing Dexora — the first open-source Vision-Language-Action system for dual-arm, dual-hand, 36-DoF dexterous manipulation. 🦾 Dual Arms 🖐️ Dual Hands 🎯 36 DoF Control 🌍 Open Source Trained on: • 100K simulated trajectories • 10K real-world demonstrations Dexora achieves: ✓ 90%+ success on basic manipulation ✓ Strong dexterous manipulation performance ✓ Cross-embodiment generalization Our key hypothesis: Train on the hardest embodiment. Transfer to simpler robots later. Instead of scaling up gripper policies, we train directly in the most expressive action space and project downward to simpler embodiments. This may be a practical path toward universal robot controllers. 🎥 Demos: 📄 Paper:

Hao Zhao

17,474 Aufrufe • vor 4 Monaten

─Fact: Ex Machina is out of tune (F/F#) but aside of a stylistic device, it’s actually meant to represent the limbo b/w the human and robotic. Yves sings the correct key cause she's the robot, perfection. The instrumental is out of tune cause it represents the human, imperfect.

─Fact: Ex Machina is out of tune (F/F#) but aside of a stylistic device, it’s actually meant to represent the limbo b/w the human and robotic. Yves sings the correct key cause she's the robot, perfection. The instrumental is out of tune cause it represents the human, imperfect.

Xiomi♮

32,288 Aufrufe • vor 6 Monaten