Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Teaching bimanual robot hands to perform very complex tasks has been notoriously challenging. In our work, Bidex: Bimanual Dexterity for Complex Tasks, we’ve developed a low-cost system that completes a wide range of highly dexterous tasks in real-time.

Kenny Shaw

1,393 subscribers

40,125 Aufrufe • vor 1 Jahr •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Kenny Shaw

Kenny Shawvor 1 Jahr

Bidex uses a teacher arm connected to a motion capture glove worn by a teleoperator. The teacher arm data is mirrored to any standard robot arm and the motion capture glove data is retargeted to our 21DOF LEAP Hand. This setup is lightweight, portable, precise, and user-friendly.

Profilbild von Kenny Shaw

Kenny Shawvor 1 Jahr

Bidex can perform many tasks from peeling a cucumber to spraying of cleaning liquids to mobile tasks such as picking up trash. Compared to the vision-based Apple Vision Pro, we find in our user study that Bidex is faster, more intuitive and leads to higher success rates.

Profilbild von Kenny Shaw

Kenny Shawvor 1 Jahr

Thanks to Bidex's advanced capabilities, collecting high-quality data for behavior cloning is very efficient. The teleoperation data is smoother which results in better policies using less data. We pre-train these policies on internet videos to aid in generalization.

Profilbild von Kenny Shaw

Kenny Shawvor 1 Jahr

The complete Bidex, which includes two robot arms ($16k), teacher arms ($500), robot hands ($6k), and gloves ($6k), costs under $30,000. We use LEAP Hand v2, a 21 DOF hybrid rigid-soft hand that is compliant yet very strong, easy to assemble, and has human-like kinematics.

Profilbild von Kenny Shaw

Kenny Shawvor 1 Jahr

The robot hands and Manus glove code are open sourced at Bidex works with many robot arms! #Corl2024 @ManusMeta It was led by me and @yulongli42 with @Jiahui_Yang6709 @mohansrirama Ray Liu, @Haoyu_Xiong_ and advisors @mendonca_rl and @pathak2206

Profilbild von Kenny Shaw

Kenny Shawvor 1 Jahr

@yulongli42 @Jiahui_Yang6709 @mohansrirama @Haoyu_Xiong_ @mendonca_rl @pathak2206 Bidex is portable so it is easy to teleoperate from anywhere! In future work, we are looking at highly dexterous mobile tasks in-the-wild. Here is a quick preview. :)

Profilbild von Brian Smith

Brian Smithvor 1 Jahr

Congrats on the launch!

Profilbild von Social Hardware International

Social Hardware Internationalvor 1 Jahr

Phenomenal work! 🤖👐 Teaching bimanual robot hands to handle complex tasks is a breakthrough for robotics. Real-time dexterity has the potential to revolutionize industries, from healthcare to waste management. At Social Hardware, we’re focusing on tele-operational systems for hazardous tasks, ensuring both precision and worker safety. It's inspiring to see advancements like Bidex contributing to the evolution of robotics! Looking forward to seeing more. 🚀 #Robotics #TeleOperation #Innovation

Profilbild von Chaoyi Pan

Chaoyi Panvor 1 Jahr

Awesome! How do you transfer across different hands? I guess that would be quite challenging.

Profilbild von Kenny Shaw

Kenny Shawvor 1 Jahr

Yes, we released this code on our website. In short, we optimize for matching pinch grasp similarity between the human and robot hands and optimize for fingertip positions. The concept is based our older works such as Robotic Telekinesis.

Ähnliche Videos

Anyone skeptical about AI’s ability to perform diverse physical tasks should watch this. Physical Intelligence's π₀ model in action: 18 minutes of bimanual robots autonomously handling complex, dexterous chores.

Anyone skeptical about AI’s ability to perform diverse physical tasks should watch this. Physical Intelligence's π₀ model in action: 18 minutes of bimanual robots autonomously handling complex, dexterous chores.

The Humanoid Hub

210,507 Aufrufe • vor 1 Jahr

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

Mandi Zhao

121,612 Aufrufe • vor 1 Jahr

Researchers have developed Humanoid-GPT, a new AI system that helps humanoid robots perform complex movements and tasks. Tested on the Unitree G1 robot, it enables real-time whole-body control and can perform new actions without special training. During demonstrations, the robot successfully carried out a variety of movements and tasks, showing strong potential for real-world applications.

Researchers have developed Humanoid-GPT, a new AI system that helps humanoid robots perform complex movements and tasks. Tested on the Unitree G1 robot, it enables real-time whole-body control and can perform new actions without special training. During demonstrations, the robot successfully carried out a variety of movements and tasks, showing strong potential for real-world applications.

Space and Technology

22,234 Aufrufe • vor 2 Monaten

Low-cost teleop systems have democratized robot data collection, but they lack any force feedback, making it challenging to teleoperate contact-rich tasks. Many robot arms provide force information — a critical yet underutilized modality in robot learning. We introduce: 1. 🦾A low-cost, force-feedback-enabled teleop system. 2. 🥊Force-Attending Curriculum Training (FACTR) uses force to improve generalization in complex, contact-rich tasks. 🧵(1/N)

Low-cost teleop systems have democratized robot data collection, but they lack any force feedback, making it challenging to teleoperate contact-rich tasks. Many robot arms provide force information — a critical yet underutilized modality in robot learning. We introduce: 1. 🦾A low-cost, force-feedback-enabled teleop system. 2. 🥊Force-Attending Curriculum Training (FACTR) uses force to improve generalization in complex, contact-rich tasks. 🧵(1/N)

Jason Liu

151,921 Aufrufe • vor 1 Jahr

Sim2Real RL for Vision-Based Dexterous Manipulation on Humanoids TLDR - we train a humanoid robot with two multifingered hands to perform a range of dexterous manipulation tasks robust generalization and high performance without human demonstration :D

Sim2Real RL for Vision-Based Dexterous Manipulation on Humanoids TLDR - we train a humanoid robot with two multifingered hands to perform a range of dexterous manipulation tasks robust generalization and high performance without human demonstration :D

Toru

49,889 Aufrufe • vor 1 Jahr

How to chain multiple dexterous skills to tackle complex long-horizon manipulation tasks? Imagine retrieving a LEGO block from a pile, rotating it in-hand, and inserting it at the desired location to build a structure. Introducing our new work - Sequential Dexterity 🧵👇

How to chain multiple dexterous skills to tackle complex long-horizon manipulation tasks? Imagine retrieving a LEGO block from a pile, rotating it in-hand, and inserting it at the desired location to build a structure. Introducing our new work - Sequential Dexterity 🧵👇

Chen Wang

159,809 Aufrufe • vor 3 Jahren

We’ve developed a memory system for our models that provides both short-term visual memory and long-term semantic memory. Our approach allows us to train robots to perform long and complex tasks, like cleaning up a kitchen or preparing a grilled cheese sandwich from scratch 👇

We’ve developed a memory system for our models that provides both short-term visual memory and long-term semantic memory. Our approach allows us to train robots to perform long and complex tasks, like cleaning up a kitchen or preparing a grilled cheese sandwich from scratch 👇

Physical Intelligence

458,339 Aufrufe • vor 6 Monaten

Our newest model, π0.7, has some interesting emergent capabilities: it can control a new robot to fold shirts for which we had no shirt folding data, figure out how to use an appliance with language-based coaching, and perform a wide range of dexterous tasks all in one model!

Our newest model, π0.7, has some interesting emergent capabilities: it can control a new robot to fold shirts for which we had no shirt folding data, figure out how to use an appliance with language-based coaching, and perform a wide range of dexterous tasks all in one model!

Physical Intelligence

536,996 Aufrufe • vor 4 Monaten

Very cool to see how robots are now mastering these tasks, but as I read Benjie's blog ( I’m even more impressed by our human ingenuity in translating human dexterity into robot-specific behaviors (by the demonstrators): - wrapping sock on gripper to invert it - mashing plastic bag on table to open it - key reorientation by multiple bimanual handovers, etc Perhaps the next “Olympics” should shift from doing the most challenging tasks to replicating such human ingenuity at test time?

Very cool to see how robots are now mastering these tasks, but as I read Benjie's blog ( I’m even more impressed by our human ingenuity in translating human dexterity into robot-specific behaviors (by the demonstrators): - wrapping sock on gripper to invert it - mashing plastic bag on table to open it - key reorientation by multiple bimanual handovers, etc Perhaps the next “Olympics” should shift from doing the most challenging tasks to replicating such human ingenuity at test time?

Wenlong Huang

20,552 Aufrufe • vor 7 Monaten

Learning about active perception with Haoyu Xiong -- your robot needs a head, and to be able to control where it's looking, in order to perform complex tasks!

Learning about active perception with Haoyu Xiong -- your robot needs a head, and to be able to control where it's looking, in order to perform complex tasks!

Chris Paxton

22,784 Aufrufe • vor 1 Jahr

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Max Fu

40,466 Aufrufe • vor 2 Jahren

Scaling vision-language-action (VLA) models to high-DoF dexterous hands has long been a "holy grail" challenge due to the high-dimensional action space and data scarcity. As a wrap up of the year 2025, we are releasing GR-Dexter, a holistic hardware-model-data framework for generalist manipulation on a bimanual dexterous-hand robot. This is the first VLA system to achieve: ✅ High-DoF Control: Managing a 56-DoF bimanual system (21-DoF per hand). ✅ Long-Horizon Tasks with tool use: Vacuuming, bread serving with tongs, and table decluttering. ✅ Open-World Generalization: Robust performance with unseen objects and abstract instructions. Project page: ArXiv:

Scaling vision-language-action (VLA) models to high-DoF dexterous hands has long been a "holy grail" challenge due to the high-dimensional action space and data scarcity. As a wrap up of the year 2025, we are releasing GR-Dexter, a holistic hardware-model-data framework for generalist manipulation on a bimanual dexterous-hand robot. This is the first VLA system to achieve: ✅ High-DoF Control: Managing a 56-DoF bimanual system (21-DoF per hand). ✅ Long-Horizon Tasks with tool use: Vacuuming, bread serving with tongs, and table decluttering. ✅ Open-World Generalization: Robust performance with unseen objects and abstract instructions. Project page: ArXiv:

Xiao Ma

94,056 Aufrufe • vor 8 Monaten

Introducing /boost in Google Antigravity. The Antigravity harness can tackle a wide range of tasks right out of the box, but /boost helps if you are tackling particularly complex tasks and are willing to spend some more tokens to push the system to reason deeply about the challenge at hand. Here is how it works ⬇️

Introducing /boost in Google Antigravity. The Antigravity harness can tackle a wide range of tasks right out of the box, but /boost helps if you are tackling particularly complex tasks and are willing to spend some more tokens to push the system to reason deeply about the challenge at hand. Here is how it works ⬇️

Google Antigravity

197,656 Aufrufe • vor 12 Tagen

Tesla Optimus can arrange batteries in their factories, ours can do skincare (on Yuzhe Qin)! We opensource Bunny-VisionPro, a teleoperation system for bimanual hand manipulation. The users can control the robot hands in real time using VisionPro, flexible like a bunny. 🐇 We also have kitchen tasks, playing Rubik's Cube, and dynamic motion tasks. Imitation learning policies are trained on sweeping with a broom, serving a drink, and wiping glasses. Check our website for more details: The project is led by Runyu Ding Runyu Ding, Yuzhe Qin Yuzhe Qin , and Jiyue Zhu

Tesla Optimus can arrange batteries in their factories, ours can do skincare (on Yuzhe Qin)! We opensource Bunny-VisionPro, a teleoperation system for bimanual hand manipulation. The users can control the robot hands in real time using VisionPro, flexible like a bunny. 🐇 We also have kitchen tasks, playing Rubik's Cube, and dynamic motion tasks. Imitation learning policies are trained on sweeping with a broom, serving a drink, and wiping glasses. Check our website for more details: The project is led by Runyu Ding Runyu Ding, Yuzhe Qin Yuzhe Qin , and Jiyue Zhu

Xiaolong Wang

90,938 Aufrufe • vor 2 Jahren

Agility Robotics is focusing on warehouse logistics. They'll be competing with wheeled bimanual robots that are highly efficient in pick-and-place tasks. One example is Reflex Robotics, a NY-based startup that recently partnered with GXO to deploy robots in warehouses.

Agility Robotics is focusing on warehouse logistics. They'll be competing with wheeled bimanual robots that are highly efficient in pick-and-place tasks. One example is Reflex Robotics, a NY-based startup that recently partnered with GXO to deploy robots in warehouses.

The Humanoid Hub

36,235 Aufrufe • vor 1 Jahr

This robodog aims to remove humans from hazardous work environments The Robot Teleoperativo 2 project out of the Dynamic Legged Systems lab has developed an advanced teleoperation system designed to operate in high-risk environments, minimizing dangers to human workers. By integrating cutting-edge technologies in tele-locomotion, tele-manipulation, and remote human-robot interaction, the system enables precise, dual-arm control for complex tasks in hazardous conditions.

This robodog aims to remove humans from hazardous work environments The Robot Teleoperativo 2 project out of the Dynamic Legged Systems lab has developed an advanced teleoperation system designed to operate in high-risk environments, minimizing dangers to human workers. By integrating cutting-edge technologies in tele-locomotion, tele-manipulation, and remote human-robot interaction, the system enables precise, dual-arm control for complex tasks in hazardous conditions.

Circuit

22,831 Aufrufe • vor 1 Jahr