Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

It’s been a massive week for embodied AI foundation models: the pace of this field is truly staggering. Throwing it back to the SimToolReal work by Kushal and Tyler Lum (Cornell & Stanford labs). In February, they achieved zero-shot tool manipulation across 24 tasks using a single RL policy...

SharpaRobotics's profile picture

Sharpa

5,021 subscribers

868,354 Aufrufe • vor 4 Monaten •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.
1:52

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.

Humanoids daily

12,391 Aufrufe • vor 5 Monaten

Evaluation is a critical bottleneck in building robot foundation models. Check out our latest work RoboLab, led by Jenai Xuning Yang, which addresses this exact challenge. Its a high-fidelity simulation environment for testing these models. A truly generalist policy should be able to complete these tasks zero-shot, and this benchmark highlights exactly how far we still have to go. More info 👇
0:26

Evaluation is a critical bottleneck in building robot foundation models. Check out our latest work RoboLab, led by Jenai Xuning Yang, which addresses this exact challenge. Its a high-fidelity simulation environment for testing these models. A truly generalist policy should be able to complete these tasks zero-shot, and this benchmark highlights exactly how far we still have to go. More info 👇

Ankit Goyal

29,981 Aufrufe • vor 4 Monaten

This looks like another step toward removing the data bottleneck in robotics. The bottleneck exists because recording demonstrations is easy, but labeling every grasp, hold, move, and release is slow and expensive. Perceptron Egocentric dropped a new robotics video annotation system. You give it raw robot-camera or first-person/egocentric video. It gives back structured labels that look more like training data: where each small manipulation starts and ends, what the subtask is, what each hand is doing, where the hands are frame by frame, and which hand is left or right. So, Input: raw video of a person or robot doing a task. Output: a machine-readable breakdown of the physical actions inside that video. Basically, it is a machine that converts robot video into policy-training supervision. That means timestamps, subtask boundaries, per-hand actions, and left-right hand grounding. So this will turn large piles of robot or egocentric video into supervision for training robot policies.
0:58

This looks like another step toward removing the data bottleneck in robotics. The bottleneck exists because recording demonstrations is easy, but labeling every grasp, hold, move, and release is slow and expensive. Perceptron Egocentric dropped a new robotics video annotation system. You give it raw robot-camera or first-person/egocentric video. It gives back structured labels that look more like training data: where each small manipulation starts and ends, what the subtask is, what each hand is doing, where the hands are frame by frame, and which hand is left or right. So, Input: raw video of a person or robot doing a task. Output: a machine-readable breakdown of the physical actions inside that video. Basically, it is a machine that converts robot video into policy-training supervision. That means timestamps, subtask boundaries, per-hand actions, and left-right hand grounding. So this will turn large piles of robot or egocentric video into supervision for training robot policies.

Rohan Paul

18,903 Aufrufe • vor 2 Monaten

This is a demonstration video of in-hand object rotation with new Torobo Hand. By leveraging large-scale parallel reinforcement learning in NVIDIA Isaac Sim, the learned policy can perform object manipulation in MuJoCo and on the real robot without any additional training.
1:09

This is a demonstration video of in-hand object rotation with new Torobo Hand. By leveraging large-scale parallel reinforcement learning in NVIDIA Isaac Sim, the learned policy can perform object manipulation in MuJoCo and on the real robot without any additional training.

東京ロボティクス株式会社

10,909 Aufrufe • vor 1 Jahr

Teaching robots to perform dexterous manipulation tasks currently requires teleoperation, which limits demonstration quality, speed, and scalability. Instead, why not use human videos? The problem is that a human hand isn’t a robot hand, so data must be retargeted using simulation to resolve issues like collisions and interpenetration when controlling the hand. In VideoManip, Hongyi Chen and co-authors built a system to solve this problem, taking in RGB videos of humans performing manipulation tasks and using them to create accurate simulations with which to learn robot policies. Watch episode #73 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton, now to learn more!
44:20

Teaching robots to perform dexterous manipulation tasks currently requires teleoperation, which limits demonstration quality, speed, and scalability. Instead, why not use human videos? The problem is that a human hand isn’t a robot hand, so data must be retargeted using simulation to resolve issues like collisions and interpenetration when controlling the hand. In VideoManip, Hongyi Chen and co-authors built a system to solve this problem, taking in RGB videos of humans performing manipulation tasks and using them to create accurate simulations with which to learn robot policies. Watch episode #73 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton, now to learn more!

RoboPapers

27,360 Aufrufe • vor 4 Monaten

The Dublin Arm was invented in 1921 and was a significant advancement in artificial limb technology. It operated using a Bowden cable mechanism, where cables connected to the user's residual limb enabled hand or hook movements. By contracting specific muscles, users could control the cables, allowing them to grasp objects and perform various tasks with improved dexterity. This pioneering prosthetic limb set the stage for future advancements in the field.
0:51

The Dublin Arm was invented in 1921 and was a significant advancement in artificial limb technology. It operated using a Bowden cable mechanism, where cables connected to the user's residual limb enabled hand or hook movements. By contracting specific muscles, users could control the cables, allowing them to grasp objects and perform various tasks with improved dexterity. This pioneering prosthetic limb set the stage for future advancements in the field.

Historic Vids

1,830,102 Aufrufe • vor 3 Jahren

Foundation models are enough to solve robotics! Unfortunately, this is not true. We keep hearing that Vision-Language-Action (VLA) models struggle because of the gap between static training and the dynamic real world. A German startup (Sereact) just released a solution that bridges this gap perfectly. They are introducing a new paradigm called Interactive RL Policy Patching. It's a distributed framework that allows robots to self-learn from human corrections without needing full retraining. When a robot fails, a human operator provides a brief "patch" or demonstration. The system then uses online off-policy reinforcement learning to update the behavior instantly. This is powered by a massive foundation model trained on hundreds of millions of interactions from over 100 deployed robot stations. The best part is the distributed parameter synchronization... When one robot learns a fix, the update is published fleet-wide... meaning the entire swarm gets smarter from a single human intervention. They are already proving this on complex manipulation tasks like shoe unboxing and screw sorting, drastically reducing the data needed to handle edge cases. Real-world environments are unforgiving, and I love seeing systems that can actually adapt on the fly! 📍 More info:
0:38

Foundation models are enough to solve robotics! Unfortunately, this is not true. We keep hearing that Vision-Language-Action (VLA) models struggle because of the gap between static training and the dynamic real world. A German startup (Sereact) just released a solution that bridges this gap perfectly. They are introducing a new paradigm called Interactive RL Policy Patching. It's a distributed framework that allows robots to self-learn from human corrections without needing full retraining. When a robot fails, a human operator provides a brief "patch" or demonstration. The system then uses online off-policy reinforcement learning to update the behavior instantly. This is powered by a massive foundation model trained on hundreds of millions of interactions from over 100 deployed robot stations. The best part is the distributed parameter synchronization... When one robot learns a fix, the update is published fleet-wide... meaning the entire swarm gets smarter from a single human intervention. They are already proving this on complex manipulation tasks like shoe unboxing and screw sorting, drastically reducing the data needed to handle edge cases. Real-world environments are unforgiving, and I love seeing systems that can actually adapt on the fly! 📍 More info:

Ilir Aliu

18,210 Aufrufe • vor 7 Monaten

🧐Applying world models to improve real-world policy on challenging manipulation tasks used to be considered out of reach. 😌After sustained effort, we’re now seeing encouraging progress. 🚀Thrilled to introduce RISE: Self-Improving Robot Policy with Compositional World Model RISE is, to our knowledge, the first work to use a world model as an effective learning environment for challenging real-world manipulation, enabling policy improvement on tasks that demand high dynamics, dexterity, and precision. Incredible teamwork with KL Hongyang Li Yue Xiangyu Hao Zhao Chonghao Sima
4:04

🧐Applying world models to improve real-world policy on challenging manipulation tasks used to be considered out of reach. 😌After sustained effort, we’re now seeing encouraging progress. 🚀Thrilled to introduce RISE: Self-Improving Robot Policy with Compositional World Model RISE is, to our knowledge, the first work to use a world model as an effective learning environment for challenging real-world manipulation, enabling policy improvement on tasks that demand high dynamics, dexterity, and precision. Incredible teamwork with KL Hongyang Li Yue Xiangyu Hao Zhao Chonghao Sima

Jiazhi Yang

77,052 Aufrufe • vor 7 Monaten

Xynova just launched Flex 2, its second-gen hybrid dexterous hand — and Xiaomi is already on the cap table. 🤖✋ The spec sheet is sharp: 23 DOF, 400g palm, ±0.1mm repeatability, 0.05N force control, 12kg single-hand grasp load, and multimodal sensing for adaptive grasping and slip detection. Xynova is not only building a robotic hand. It is building the manipulation stack around it: micro linear actuators, joint modules, sensing, control, and developer-facing integration. The Xiaomi link is confirmed. Xiaomi Strategic Investment joined Xynova’s angel round and continued adding in the Pre-A round. What is not confirmed: whether Xiaomi’s own robot hand uses Xynova’s technology. That detail still has no public proof. But the direction is clear: Xiaomi is putting money into the component layer that decides whether humanoids can actually handle objects, tools, and daily tasks. This is the part of robotics that rarely looks flashy on stage. But it decides what a robot can really do with its hands.
0:35

Xynova just launched Flex 2, its second-gen hybrid dexterous hand — and Xiaomi is already on the cap table. 🤖✋ The spec sheet is sharp: 23 DOF, 400g palm, ±0.1mm repeatability, 0.05N force control, 12kg single-hand grasp load, and multimodal sensing for adaptive grasping and slip detection. Xynova is not only building a robotic hand. It is building the manipulation stack around it: micro linear actuators, joint modules, sensing, control, and developer-facing integration. The Xiaomi link is confirmed. Xiaomi Strategic Investment joined Xynova’s angel round and continued adding in the Pre-A round. What is not confirmed: whether Xiaomi’s own robot hand uses Xynova’s technology. That detail still has no public proof. But the direction is clear: Xiaomi is putting money into the component layer that decides whether humanoids can actually handle objects, tools, and daily tasks. This is the part of robotics that rarely looks flashy on stage. But it decides what a robot can really do with its hands.

RoboHub🤖

71,176 Aufrufe • vor 4 Monaten

We benchmarked leading multimodal foundation models (GPT-4o, Claude 3.5 Sonnet, Gemini, Llama, etc.) on standard computer vision tasks—from segmentation to surface normal estimation—using standard datasets like COCO and ImageNet. These models have made remarkable progress; however, it is unclear exactly where they stand in terms of understanding vision in detail. Especially when it comes to tasks beyond question-answering. How well do they understand an object's segments or geometry? Our analyses yield an assessment that is quantitatively and qualitatively detailed and is compatible with evaluations developed in the field of computer vision over the past decades. Observed trends: 🔹 The foundation models consistently underperform task-specific SOTA models across all tasks. However, they are respectable generalists, which is remarkable as they are presumably trained primarily on image-text-based tasks. 🔹 They perform semantic tasks notably better than geometric ones. 🔹 GPT-4o performs the best among non-reasoning models, getting the top position in 4 out of 6 tasks. 🔹 Reasoning models, e.g., o3, show improvements in geometric tasks. 🔹 The 'image generation' models, e.g., GPT-40 Image Generation, which have been natively trained multimodally, exhibit quirks. E.g., hallucinated objects, misalignment between the input and output, etc. 🔹 While the prompting techniques affect performance, better models exhibit less sensitivity to variations in prompts. We control for the variance introduced by the prompting methods in our experiments. 🌐 Detailed analyses, visualizations: ⌨️ code: 🧵 1/n
2:36

We benchmarked leading multimodal foundation models (GPT-4o, Claude 3.5 Sonnet, Gemini, Llama, etc.) on standard computer vision tasks—from segmentation to surface normal estimation—using standard datasets like COCO and ImageNet. These models have made remarkable progress; however, it is unclear exactly where they stand in terms of understanding vision in detail. Especially when it comes to tasks beyond question-answering. How well do they understand an object's segments or geometry? Our analyses yield an assessment that is quantitatively and qualitatively detailed and is compatible with evaluations developed in the field of computer vision over the past decades. Observed trends: 🔹 The foundation models consistently underperform task-specific SOTA models across all tasks. However, they are respectable generalists, which is remarkable as they are presumably trained primarily on image-text-based tasks. 🔹 They perform semantic tasks notably better than geometric ones. 🔹 GPT-4o performs the best among non-reasoning models, getting the top position in 4 out of 6 tasks. 🔹 Reasoning models, e.g., o3, show improvements in geometric tasks. 🔹 The 'image generation' models, e.g., GPT-40 Image Generation, which have been natively trained multimodally, exhibit quirks. E.g., hallucinated objects, misalignment between the input and output, etc. 🔹 While the prompting techniques affect performance, better models exhibit less sensitivity to variations in prompts. We control for the variance introduced by the prompting methods in our experiments. 🌐 Detailed analyses, visualizations: ⌨️ code: 🧵 1/n

Amir Zamir

73,244 Aufrufe • vor 1 Jahr

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork
0:20

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

Pascal Bornet

186,712 Aufrufe • vor 5 Monaten

Elon Musk: The hand is extremely complex. There are 50 actuators in the hand and forearm Actuator is the motor, gearbox, and power electronics. so that's 100 per robot. There's a lot of complexity So, in order to do dextrous tasks, you have to have a hand with the sensitivity, precision, and degrees of freedom of a human hand. Something we find easy to do, like pick up a screwdriver or turn a wrench, or even say thread a needle, or play the guitar, actually require a lot of dexterity
1:20

Elon Musk: The hand is extremely complex. There are 50 actuators in the hand and forearm Actuator is the motor, gearbox, and power electronics. so that's 100 per robot. There's a lot of complexity So, in order to do dextrous tasks, you have to have a hand with the sensitivity, precision, and degrees of freedom of a human hand. Something we find easy to do, like pick up a screwdriver or turn a wrench, or even say thread a needle, or play the guitar, actually require a lot of dexterity

X Freeze

66,258 Aufrufe • vor 10 Monaten

Few things to take note of with these spring training receiving reps by Tony Wolters a while back… 1. The timing. THIS is what being on time looks like. 2. The speed. THIS is what moving it fast to or through the catch looks like. 3. The presentation. THIS is what… (Cont.)
0:41

Few things to take note of with these spring training receiving reps by Tony Wolters a while back… 1. The timing. THIS is what being on time looks like. 2. The speed. THIS is what moving it fast to or through the catch looks like. 3. The presentation. THIS is what… (Cont.)

The Catching Guy

58,201 Aufrufe • vor 1 Jahr

Open-Source Robotic Hand That Costs Just $300. California-based company TetherIA.ai has officially launched its open-source robotic hand, the Aero Hand Open. Designed to be the most capable at the lowest cost, this hand is a new tool for embodied AI research. In the clip, it demonstrates impressive dexterity by grasping an M5 screw, picking up an iPhone, and opening a soda can. Key Features: ► Price: $314.00 ► Open Source: CAD, electronics, and firmware are all open source. ► Actuation: 7 Active Degrees of Freedom (16 joints) with a tendon-driven design. ► Weight: Under 400 grams. ► Purpose: Built for embodied AI research to make dexterous manipulation accessible to all. For more details and to purchase:
2:12

Open-Source Robotic Hand That Costs Just $300. California-based company TetherIA.ai has officially launched its open-source robotic hand, the Aero Hand Open. Designed to be the most capable at the lowest cost, this hand is a new tool for embodied AI research. In the clip, it demonstrates impressive dexterity by grasping an M5 screw, picking up an iPhone, and opening a soda can. Key Features: ► Price: $314.00 ► Open Source: CAD, electronics, and firmware are all open source. ► Actuation: 7 Active Degrees of Freedom (16 joints) with a tendon-driven design. ► Weight: Under 400 grams. ► Purpose: Built for embodied AI research to make dexterous manipulation accessible to all. For more details and to purchase:

RoboHub🤖

45,830 Aufrufe • vor 11 Monaten

⚡️EgoVerse is a first-of-its-kind, collaborative ecosystem for human-to-robot learning. The consortium leverages Project Aria to capture high-fidelity, egocentric human data — including 3D hand and head poses — to train next-gen robot manipulation policies. With over 1,300 hours of data across 2,000+ tasks, EgoVerse is a prime example of how the Aria Research Kit is being used by our partners to accelerate the future of embodied AI. Learn more: 🔗 📰 Apply for the Aria Research Kit: #MachineLearning #Robotics #ProjectAria #EgoVerse #ComputerVision Simar Kareer , Ryan Punamiya , Roger Qiu, Xiongyi Cai , Alexey Gavryushin
0:35

⚡️EgoVerse is a first-of-its-kind, collaborative ecosystem for human-to-robot learning. The consortium leverages Project Aria to capture high-fidelity, egocentric human data — including 3D hand and head poses — to train next-gen robot manipulation policies. With over 1,300 hours of data across 2,000+ tasks, EgoVerse is a prime example of how the Aria Research Kit is being used by our partners to accelerate the future of embodied AI. Learn more: 🔗 📰 Apply for the Aria Research Kit: #MachineLearning #Robotics #ProjectAria #EgoVerse #ComputerVision Simar Kareer , Ryan Punamiya , Roger Qiu, Xiongyi Cai , Alexey Gavryushin

Project Aria @Meta

18,110 Aufrufe • vor 4 Monaten

Happy to share what I’ve been working on since joining Genesis! GENE-26.5 is a one-of-a-kind, robotics-native multimodal foundation model that learns from diverse, in-the-wild data across modalities and outputs actions enabling a 54-DoF robot system to perform the most dexterous, long-horizon manipulation tasks to date—approaching human-level capability. This is the result of innovations across the full stack—data collection and processing, robot systems, model architecture, training strategies, and scalable evaluation infrastructure.
2:23

Happy to share what I’ve been working on since joining Genesis! GENE-26.5 is a one-of-a-kind, robotics-native multimodal foundation model that learns from diverse, in-the-wild data across modalities and outputs actions enabling a 54-DoF robot system to perform the most dexterous, long-horizon manipulation tasks to date—approaching human-level capability. This is the result of innovations across the full stack—data collection and processing, robot systems, model architecture, training strategies, and scalable evaluation infrastructure.

Zu Wang

20,159 Aufrufe • vor 4 Monaten

Introducing MARS, the first Personal AI Robot. In technology, getting to the future we want starts with building it together. This is why we designed Mars. Inspired by early PCs, Mars is a powerful, complete, extendable robot. It runs BASIC, an open embodied AI agent we created for you to program with code, demonstrations, and prompts. This allows Mars to perform complex long-horizon tasks involving spatial memory, reasoning, manipulation and navigation. Pre-orders open now ⬇️ Demos & details in the thread
1:10

Introducing MARS, the first Personal AI Robot. In technology, getting to the future we want starts with building it together. This is why we designed Mars. Inspired by early PCs, Mars is a powerful, complete, extendable robot. It runs BASIC, an open embodied AI agent we created for you to program with code, demonstrations, and prompts. This allows Mars to perform complex long-horizon tasks involving spatial memory, reasoning, manipulation and navigation. Pre-orders open now ⬇️ Demos & details in the thread

Axel

314,234 Aufrufe • vor 1 Jahr

We believe we’re the first robotics company to demonstrate a robot peeling an apple with dual dexterous human-like hands. This breakthrough closes a key gap in robotics, achieving bimanual, contact-rich manipulation and moving far beyond the limits of simple grippers. 🧵↓ Today’s AI models (VLMs) are excellent at perception but struggle with action. Controlling high-degree-of-freedom hands for tasks like this is incredibly complex, and precise finger-level teleoperation is nearly impossible for humans. Our first step was a shared-autonomy system: rather than controlling every finger, the operator triggers pre-learned skills like a “rotate apple or tennis ball” primitive via a keyboard press or pedal. This makes scalable data collection and RL training possible. How does the AI manage this? We created "MoDE-VLA" (Mixture of Dexterous Experts). It fuses vision, language, force, and touch data by using a team of specialist "experts," making control in high-dimensional spaces stable and effective. The combination of these two innovations allows for seamless, contact-rich manipulation. The human provides high-level guidance, and the robot executes the complex in-hand coordination required. This work paves the way for robots that can safely handle delicate tasks in human environments. Want the full technical details? 📄 Read the full research paper: Visit us at NVIDIA GTC Booth #1838, Hall 3 to learn more! #Robotics #AI #DexterousManipulation #VLA #NVIDIAGTC Nancy Villicaña NVIDIA GTC
2:51

We believe we’re the first robotics company to demonstrate a robot peeling an apple with dual dexterous human-like hands. This breakthrough closes a key gap in robotics, achieving bimanual, contact-rich manipulation and moving far beyond the limits of simple grippers. 🧵↓ Today’s AI models (VLMs) are excellent at perception but struggle with action. Controlling high-degree-of-freedom hands for tasks like this is incredibly complex, and precise finger-level teleoperation is nearly impossible for humans. Our first step was a shared-autonomy system: rather than controlling every finger, the operator triggers pre-learned skills like a “rotate apple or tennis ball” primitive via a keyboard press or pedal. This makes scalable data collection and RL training possible. How does the AI manage this? We created "MoDE-VLA" (Mixture of Dexterous Experts). It fuses vision, language, force, and touch data by using a team of specialist "experts," making control in high-dimensional spaces stable and effective. The combination of these two innovations allows for seamless, contact-rich manipulation. The human provides high-level guidance, and the robot executes the complex in-hand coordination required. This work paves the way for robots that can safely handle delicate tasks in human environments. Want the full technical details? 📄 Read the full research paper: Visit us at NVIDIA GTC Booth #1838, Hall 3 to learn more! #Robotics #AI #DexterousManipulation #VLA #NVIDIAGTC Nancy Villicaña NVIDIA GTC

Sharpa

20,429 Aufrufe • vor 6 Monaten

These guys trained a robot with water bottles, and the robot learned to pick up other objects it had never practiced on. They did this using reinforcement learning: 1. The robot had to pick a bottle out of a messy tote 2. Then it had to hand the bottle to a person 3. Only water bottles. Nothing else. After they finished training, they tested the robot on different objects: • On bottles, the success went from 80% to 98% • On snack cans, the robot's throughput rose 40% • On soft bags, the robot's performance improved by 13% even though they're deformable and much harder to handle than rigid objects. Snack cans and soft bags have different shapes and weights, and the robot was never trained on them. This was just one of three use cases they tested. Every hour we spend teaching a robot costs a ton of money. If they only get better at handling the one object they practice on, reinforcement learning will never be worth doing. But if the gains carry over, we could focus on teaching the hardest possible tasks and let the robot apply that knowledge elsewhere. Here is the full write-up by Humanoid on the infrastructure they used and what they learned: #ad
1:40

These guys trained a robot with water bottles, and the robot learned to pick up other objects it had never practiced on. They did this using reinforcement learning: 1. The robot had to pick a bottle out of a messy tote 2. Then it had to hand the bottle to a person 3. Only water bottles. Nothing else. After they finished training, they tested the robot on different objects: • On bottles, the success went from 80% to 98% • On snack cans, the robot's throughput rose 40% • On soft bags, the robot's performance improved by 13% even though they're deformable and much harder to handle than rigid objects. Snack cans and soft bags have different shapes and weights, and the robot was never trained on them. This was just one of three use cases they tested. Every hour we spend teaching a robot costs a ton of money. If they only get better at handling the one object they practice on, reinforcement learning will never be worth doing. But if the gains carry over, we could focus on teaching the hardest possible tasks and let the robot apply that knowledge elsewhere. Here is the full write-up by Humanoid on the infrastructure they used and what they learned: #ad

Santiago

13,952 Aufrufe • vor 1 Monat