正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Five years ago, OpenAI trained a 5-fingered humanoid hand to solve a Rubik’s Cube using RL and domain randomization, pushing the boundaries of sim-to-real for fine-motor tasks. It underscored the value of creating sufficiently complex simulated worlds in which robots can learn.

The Humanoid Hub

108,250 subscribers

66,761 次观看 • 1 年前 •via X (Twitter)

科学技术教育艺术

Anya Rossi• Live Now

Private livecam show

10 条评论

andrewyu 的头像

andrewyu1 年前

badmephisto this u? @karpathy

AGIHound 的头像

AGIHound1 年前

Sim-to-real is using RL is just automation. Learning like a child in the real world is the problem to solve.

Akram Choudhary 的头像

Akram Choudhary1 年前

Can't wait to see what they can do in 2025

Judge DredD 的头像

Judge DredD1 年前

Prototype is easy Production low price 60B hand is super hard Again watch Tesla Dont miss that ever

Javier Martin 的头像

Javier Martin1 年前

Si esto es cierto, ya puedes ir buscando sitio para el robot humanoide en tu casa ¿quién va a poder resistirse a que una máquina recoja la casa, la ropa, el menaje, …? Mientras lees un libro tomando un buen café? ☕️

Fariz Babayev 的头像

Fariz Babayev1 年前

Domain randomization is key, impressive to see a 5-fingered humanoid hand solve a Rubik's Cube using RL.

darthur 的头像

darthur1 年前

Oh neat! I'm in this video.

Appy Pie 的头像

Appy Pie1 年前

Impressive! Sim-to-real advancements are shaping the future of robotics.

@amuse 的头像

@amuse1 年前

Why don’t robots have six fingers?

MarketMaverick 的头像

MarketMaverick1 年前

I always wondered what happened to this hand/development...figured it would end up in Optimus. Not sure anymore though.

相关视频

4/ Rubik’s Cube solving has been a long-standing challenging benchmark for robotic manipulation. The task requires fine-grained control under the geometric and kinematic constraints imposed by the cube itself. Prior state-of-the-art is still the single-handed solver from OpenAI’ in 2019. For the first time, we can solve a Rubik's Cube using two hands together, powered by GENE.

4/ Rubik’s Cube solving has been a long-standing challenging benchmark for robotic manipulation. The task requires fine-grained control under the geometric and kinematic constraints imposed by the cube itself. Prior state-of-the-art is still the single-handed solver from OpenAI’ in 2019. For the first time, we can solve a Rubik's Cube using two hands together, powered by GENE.

Genesis AI

46,932 次观看 • 3 个月前

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.

Humanoids daily

12,391 次观看 • 4 个月前

Bob McGrew, Former Research Chief at OpenAI: "Manipulation is the hard problem we need to solve to make humanoid robots useful, not locomotion. One video of a humanoid slowly performing tasks with its hands is worth a dozen clips of choreographed routines."

Bob McGrew, Former Research Chief at OpenAI: "Manipulation is the hard problem we need to solve to make humanoid robots useful, not locomotion. One video of a humanoid slowly performing tasks with its hands is worth a dozen clips of choreographed routines."

The Humanoid Hub

95,273 次观看 • 1 年前

Excited to share our latest work: “Bio-Inspired Plastic Neural Networks for Zero-Shot Out-of-Distribution Generalization in Complex Animal-Inspired Robots” 🪲🦎 We show that Hebbian learning outperforms LSTM-based adaptation for real-world transfer. It even works without domain randomization! It can handle: ✅ Uneven terrain ✅ Morphological damage ✅ Sim-to-real gaps

Excited to share our latest work: “Bio-Inspired Plastic Neural Networks for Zero-Shot Out-of-Distribution Generalization in Complex Animal-Inspired Robots” 🪲🦎 We show that Hebbian learning outperforms LSTM-based adaptation for real-world transfer. It even works without domain randomization! It can handle: ✅ Uneven terrain ✅ Morphological damage ✅ Sim-to-real gaps

Sebastian Risi

14,390 次观看 • 1 年前

Elon just dropped a MAJOR nugget on how Tesla is going to be training Optimus to do real world tasks. They are building an Optimus Academy, which is a large scale, dedicated real-world training facility to accelerate the development of Optimus. The Academy will deploy thousands of Optimus units, potentially 10,000 to 30,000 robots, in a controlled realistic environment where they perform self-play, experiment with tasks, iterate on behaviors, and continuously generate training data through trial and error. The Tesla bots will also run millions of simulations in Tesla’s high-fidelity physics-accurate engine, allowing Optimus to close the “sim-to-real gap” by using these real-world observations to refine and validate the simulations! “You’re actually highlighting an important limitation and difference from cars. We’ll soon have 10 million cars on the road. It’s hard to duplicate that massive training flywheel. For the robot, what we’re going to need to do is build a lot of robots and put them in kind of an Optimus Academy so they can do self-play in reality. We’re actually building that out. We can have at least 10,000 Optimus robots, maybe 20-30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator, a physics-accurate reality generator, that we made for the cars. We’ll do the same thing for the robots. We actually have done that for the robots. So you have a few tens of thousands of humanoid robots doing different tasks. You can do millions of simulated robots in the simulated world. You use the tens of thousands of robots in the real world to close the simulation to reality gap. Close the sim-to-real gap.”

Elon just dropped a MAJOR nugget on how Tesla is going to be training Optimus to do real world tasks. They are building an Optimus Academy, which is a large scale, dedicated real-world training facility to accelerate the development of Optimus. The Academy will deploy thousands of Optimus units, potentially 10,000 to 30,000 robots, in a controlled realistic environment where they perform self-play, experiment with tasks, iterate on behaviors, and continuously generate training data through trial and error. The Tesla bots will also run millions of simulations in Tesla’s high-fidelity physics-accurate engine, allowing Optimus to close the “sim-to-real gap” by using these real-world observations to refine and validate the simulations! “You’re actually highlighting an important limitation and difference from cars. We’ll soon have 10 million cars on the road. It’s hard to duplicate that massive training flywheel. For the robot, what we’re going to need to do is build a lot of robots and put them in kind of an Optimus Academy so they can do self-play in reality. We’re actually building that out. We can have at least 10,000 Optimus robots, maybe 20-30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator, a physics-accurate reality generator, that we made for the cars. We’ll do the same thing for the robots. We actually have done that for the robots. So you have a few tens of thousands of humanoid robots doing different tasks. You can do millions of simulated robots in the simulated world. You use the tens of thousands of robots in the real world to close the simulation to reality gap. Close the sim-to-real gap.”

Teslaconomics

42,563 次观看 • 6 个月前

We’ve all seen videos of humanoid robots performing single tasks that are very impressive, like dancing or karate. But training humanoid robots to perform a wide range of complex motions is difficult. GMT is a general-purpose policy which can learn a wide range of robot motions. Watch Episode #32 of RoboPapers, with Zixuan Chen , co-hosted by Michael Cho - Rbt/Acc and Chris Paxton , to learn more.

We’ve all seen videos of humanoid robots performing single tasks that are very impressive, like dancing or karate. But training humanoid robots to perform a wide range of complex motions is difficult. GMT is a general-purpose policy which can learn a wide range of robot motions. Watch Episode #32 of RoboPapers, with Zixuan Chen , co-hosted by Michael Cho - Rbt/Acc and Chris Paxton , to learn more.

RoboPapers

16,605 次观看 • 10 个月前

.Elon Musk told me he plans to build “Optimus Academy” to train an army of humanoid robots. > “millions of simulated robots in the simulated world” > “tens of thousands of robots in the real world, to close the simulation to reality gap.” Why? There’s two big differences between the way Tesla will train humanoid robots and the way it built Full Self-Driving. - FSD only requires learning three degrees of freedom: turning, accelerating, braking … and a humanoid robot has to learn how to coordinate 50+ joints. - You can sell a Tesla without FSD, which allows you to collect the millions and millions of hours of driving data needed to bootstrap your model ... but you can’t sell an Optimus that hasn’t learned how to do stuff yet. The Optimus Academy is Elon’s attempt to build the data flywheel that Tesla got for free with FSD.

.Elon Musk told me he plans to build “Optimus Academy” to train an army of humanoid robots. > “millions of simulated robots in the simulated world” > “tens of thousands of robots in the real world, to close the simulation to reality gap.” Why? There’s two big differences between the way Tesla will train humanoid robots and the way it built Full Self-Driving. - FSD only requires learning three degrees of freedom: turning, accelerating, braking … and a humanoid robot has to learn how to coordinate 50+ joints. - You can sell a Tesla without FSD, which allows you to collect the millions and millions of hours of driving data needed to bootstrap your model ... but you can’t sell an Optimus that hasn’t learned how to do stuff yet. The Optimus Academy is Elon’s attempt to build the data flywheel that Tesla got for free with FSD.

Dwarkesh Patel

544,749 次观看 • 6 个月前

Teaching robots to perform dexterous manipulation tasks currently requires teleoperation, which limits demonstration quality, speed, and scalability. Instead, why not use human videos? The problem is that a human hand isn’t a robot hand, so data must be retargeted using simulation to resolve issues like collisions and interpenetration when controlling the hand. In VideoManip, Hongyi Chen and co-authors built a system to solve this problem, taking in RGB videos of humans performing manipulation tasks and using them to create accurate simulations with which to learn robot policies. Watch episode #73 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton, now to learn more!

Teaching robots to perform dexterous manipulation tasks currently requires teleoperation, which limits demonstration quality, speed, and scalability. Instead, why not use human videos? The problem is that a human hand isn’t a robot hand, so data must be retargeted using simulation to resolve issues like collisions and interpenetration when controlling the hand. In VideoManip, Hongyi Chen and co-authors built a system to solve this problem, taking in RGB videos of humans performing manipulation tasks and using them to create accurate simulations with which to learn robot policies. Watch episode #73 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton, now to learn more!

RoboPapers

27,344 次观看 • 3 个月前

Researchers have developed Humanoid-GPT, a new AI system that helps humanoid robots perform complex movements and tasks. Tested on the Unitree G1 robot, it enables real-time whole-body control and can perform new actions without special training. During demonstrations, the robot successfully carried out a variety of movements and tasks, showing strong potential for real-world applications.

Researchers have developed Humanoid-GPT, a new AI system that helps humanoid robots perform complex movements and tasks. Tested on the Unitree G1 robot, it enables real-time whole-body control and can perform new actions without special training. During demonstrations, the robot successfully carried out a variety of movements and tasks, showing strong potential for real-world applications.

Space and Technology

22,052 次观看 • 1 个月前

🚨🇺🇸FIGURE ROBOT LEARNS TO WALK IN HOURS Turns out you can teach a robot to walk—if you’ve got a few years to burn in a physics simulator (or a few hours of GPU time). Figure’s team trained a neural network using reinforcement learning to mimic natural human locomotion, and the wild part is it worked on the first try with real hardware. No tuning. No fiddling. Just plug and strut. Thanks to sim-to-real transfer and some sneaky domain randomization, the robot didn’t just wobble out of the lab—it walked like it had somewhere to be. Next up: running. Not terrifying at all. Source: Figure

🚨🇺🇸FIGURE ROBOT LEARNS TO WALK IN HOURS Turns out you can teach a robot to walk—if you’ve got a few years to burn in a physics simulator (or a few hours of GPU time). Figure’s team trained a neural network using reinforcement learning to mimic natural human locomotion, and the wild part is it worked on the first try with real hardware. No tuning. No fiddling. Just plug and strut. Thanks to sim-to-real transfer and some sneaky domain randomization, the robot didn’t just wobble out of the lab—it walked like it had somewhere to be. Next up: running. Not terrifying at all. Source: Figure

Mario Nawfal

117,866 次观看 • 1 年前

“Dexterous capability is directly proportional to the size of the addressable market for general-purpose humanoid robots,” says James Wells, interim CEO and CCO. Sanctuary AI announces its latest technical milestone using hydraulic actuation, opening up a wide range of industrial and high value work tasks that require fine manipulation. #AI #Robotics

“Dexterous capability is directly proportional to the size of the addressable market for general-purpose humanoid robots,” says James Wells, interim CEO and CCO. Sanctuary AI announces its latest technical milestone using hydraulic actuation, opening up a wide range of industrial and high value work tasks that require fine manipulation. #AI #Robotics

Sanctuary AI

16,583 次观看 • 1 年前

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

Physical Intelligence

438,768 次观看 • 4 个月前

Super excited to announce SIMA 2! It’s a general agent that can understand & reason about complex instructions and complete tasks in simulated game worlds, even ones it has never seen before. Incredible to see how it can learn just from self-play… a crucial step towards AGI

Super excited to announce SIMA 2! It’s a general agent that can understand & reason about complex instructions and complete tasks in simulated game worlds, even ones it has never seen before. Incredible to see how it can learn just from self-play… a crucial step towards AGI

Demis Hassabis

154,038 次观看 • 9 个月前

Trained a humanoid entirely in a 3D scan of the office. Zero real-world fine-tuning. It just walked in and worked. RL needs hundreds of thousands of attempts, and real robots can't afford to crash. A misjudged gap or a glass door collision breaks hardware and costs hours resetting. So you train in a sim. But sim policies usually train on randomized, untextured geometry; depth is easy to fake. The robot learns structure, not the real world: no materials, no lighting, no idea what anything actually is. RGB cameras carry all of that but training RGB policies in generic fake worlds won’t generalize to the real world. Niantic Spatial 🌎 Scaniverse reconstructs your scan of the real deployment site. One 360° camera walkthrough → photorealistic 3D Gaussian splat at metric scale → collision mesh pulled from the same reconstruction, so vision and physics match exactly. Drops straight into NVIDIA Isaac Sim/Lab, no manual conversion. Flexion simulation-first approach then seamlessly enables the training of RGB-only nav policies inside that reconstruction. With added domain randomization + large image encoders for robustness, this deploys straight to hardware. No real-world fine-tuning. Deployment: months of on-site adaptation → days. Tune into the NVIDIA livestream on 12 August to hear how these companies are closing the sim2real gap: NVIDIA Robotics ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Trained a humanoid entirely in a 3D scan of the office. Zero real-world fine-tuning. It just walked in and worked. RL needs hundreds of thousands of attempts, and real robots can't afford to crash. A misjudged gap or a glass door collision breaks hardware and costs hours resetting. So you train in a sim. But sim policies usually train on randomized, untextured geometry; depth is easy to fake. The robot learns structure, not the real world: no materials, no lighting, no idea what anything actually is. RGB cameras carry all of that but training RGB policies in generic fake worlds won’t generalize to the real world. Niantic Spatial 🌎 Scaniverse reconstructs your scan of the real deployment site. One 360° camera walkthrough → photorealistic 3D Gaussian splat at metric scale → collision mesh pulled from the same reconstruction, so vision and physics match exactly. Drops straight into NVIDIA Isaac Sim/Lab, no manual conversion. Flexion simulation-first approach then seamlessly enables the training of RGB-only nav policies inside that reconstruction. With added domain randomization + large image encoders for robustness, this deploys straight to hardware. No real-world fine-tuning. Deployment: months of on-site adaptation → days. Tune into the NVIDIA livestream on 12 August to hear how these companies are closing the sim2real gap: NVIDIA Robotics ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

117,692 次观看 • 4 天前

Elon Musk in new interview on how Tesla’s Optimus robot will learn more quickly: "What we're going to need to do is build a lot of robots and put them in a kind of like a "Optimus Academy" so they can do self-play in reality. We're actually building that out. We're gonna have at least 10,000 Optimus robots, maybe 20,000 or 30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator. A physics-accurate reality generator that we made for the cars. We'll do the same things for the robots, and actually have done that for the robots. You can do millions of simulated robots in a simulated world, and you use the tens of thousands in the real world to close the simulation to reality gap.”

Elon Musk in new interview on how Tesla’s Optimus robot will learn more quickly: "What we're going to need to do is build a lot of robots and put them in a kind of like a "Optimus Academy" so they can do self-play in reality. We're actually building that out. We're gonna have at least 10,000 Optimus robots, maybe 20,000 or 30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator. A physics-accurate reality generator that we made for the cars. We'll do the same things for the robots, and actually have done that for the robots. You can do millions of simulated robots in a simulated world, and you use the tens of thousands in the real world to close the simulation to reality gap.”

Sawyer Merritt

250,956 次观看 • 6 个月前

ELON: TESLA’S ROBOTS ARE MOVING FROM SIMULATIONS TO REALITY “For the robot, we’re going to need to build a lot of them and put them in an Optimus Academy so they can do self-play in reality and test different tasks. Tesla has quite a good, physics accurate, reality-generator, and we’re doing that for the robots. So you can do millions of robots in a simulated world, and then 10k in the real world to close the simulation to reality gap. Grok would orchestrate the behavior of the Optimus robots.” Source: Cheeky Pint Podcast, Elon Musk

ELON: TESLA’S ROBOTS ARE MOVING FROM SIMULATIONS TO REALITY “For the robot, we’re going to need to build a lot of them and put them in an Optimus Academy so they can do self-play in reality and test different tasks. Tesla has quite a good, physics accurate, reality-generator, and we’re doing that for the robots. So you can do millions of robots in a simulated world, and then 10k in the real world to close the simulation to reality gap. Grok would orchestrate the behavior of the Optimus robots.” Source: Cheeky Pint Podcast, Elon Musk

Mario Nawfal

19,289 次观看 • 6 个月前

Whether teleoperated or autonomous, whole body movements like picking objects from the ground are challenging for humanoid robots. A UC San Diego paper introduces a framework that combines sim-to-real RL and trajectory optimization for adaptive, real-time whole body control.

Whether teleoperated or autonomous, whole body movements like picking objects from the ground are challenging for humanoid robots. A UC San Diego paper introduces a framework that combines sim-to-real RL and trajectory optimization for adaptive, real-time whole body control.

The Humanoid Hub

27,879 次观看 • 1 年前

Hands are the bridge for the emergence of intelligence. DM-Hand1, developed by Shenzhen DaimonRobotics, is a humanoid hand equipped with visual and tactile sensors, enabling robots to perceive any object and its changes and respond appropriately. This means humanoid robots equipped with this hand show strong potential in precision tasks, cluttered item sorting, and especially in service roles like elderly care.

Hands are the bridge for the emergence of intelligence. DM-Hand1, developed by Shenzhen DaimonRobotics, is a humanoid hand equipped with visual and tactile sensors, enabling robots to perceive any object and its changes and respond appropriately. This means humanoid robots equipped with this hand show strong potential in precision tasks, cluttered item sorting, and especially in service roles like elderly care.

CyberRobo

97,691 次观看 • 1 年前