Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Five years ago, OpenAI trained a 5-fingered humanoid hand to solve a Rubik’s Cube using RL and domain randomization, pushing the boundaries of sim-to-real for fine-motor tasks. It underscored the value of creating sufficiently complex simulated worlds in which robots can learn.

The Humanoid Hub

108,250 subscribers

66,761 просмотров • 1 год назад •via X (Twitter)

Наука и технологии Образование Искусство

Anya Rossi• Live Now

Private livecam show

Комментарии: 10

Фото профиля andrewyu

andrewyu1 год назад

badmephisto this u? @karpathy

Фото профиля AGIHound

AGIHound1 год назад

Sim-to-real is using RL is just automation. Learning like a child in the real world is the problem to solve.

Фото профиля Akram Choudhary

Akram Choudhary1 год назад

Can't wait to see what they can do in 2025

Фото профиля Judge DredD

Judge DredD1 год назад

Prototype is easy Production low price 60B hand is super hard Again watch Tesla Dont miss that ever

Фото профиля Javier Martin

Javier Martin1 год назад

Si esto es cierto, ya puedes ir buscando sitio para el robot humanoide en tu casa ¿quién va a poder resistirse a que una máquina recoja la casa, la ropa, el menaje, …? Mientras lees un libro tomando un buen café? ☕️

Фото профиля Fariz Babayev

Fariz Babayev1 год назад

Domain randomization is key, impressive to see a 5-fingered humanoid hand solve a Rubik's Cube using RL.

Фото профиля darthur

darthur1 год назад

Oh neat! I'm in this video.

Фото профиля Appy Pie

Appy Pie1 год назад

Impressive! Sim-to-real advancements are shaping the future of robotics.

Фото профиля @amuse

@amuse1 год назад

Why don’t robots have six fingers?

Фото профиля MarketMaverick

MarketMaverick1 год назад

I always wondered what happened to this hand/development...figured it would end up in Optimus. Not sure anymore though.

Похожие видео

4/ Rubik’s Cube solving has been a long-standing challenging benchmark for robotic manipulation. The task requires fine-grained control under the geometric and kinematic constraints imposed by the cube itself. Prior state-of-the-art is still the single-handed solver from OpenAI’ in 2019. For the first time, we can solve a Rubik's Cube using two hands together, powered by GENE.

4/ Rubik’s Cube solving has been a long-standing challenging benchmark for robotic manipulation. The task requires fine-grained control under the geometric and kinematic constraints imposed by the cube itself. Prior state-of-the-art is still the single-handed solver from OpenAI’ in 2019. For the first time, we can solve a Rubik's Cube using two hands together, powered by GENE.

Genesis AI

46,932 просмотров • 3 месяцев назад

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.

Humanoids daily

12,391 просмотров • 5 месяцев назад

Bob McGrew, Former Research Chief at OpenAI: "Manipulation is the hard problem we need to solve to make humanoid robots useful, not locomotion. One video of a humanoid slowly performing tasks with its hands is worth a dozen clips of choreographed routines."

Bob McGrew, Former Research Chief at OpenAI: "Manipulation is the hard problem we need to solve to make humanoid robots useful, not locomotion. One video of a humanoid slowly performing tasks with its hands is worth a dozen clips of choreographed routines."

The Humanoid Hub

95,273 просмотров • 1 год назад

Excited to share our latest work: “Bio-Inspired Plastic Neural Networks for Zero-Shot Out-of-Distribution Generalization in Complex Animal-Inspired Robots” 🪲🦎 We show that Hebbian learning outperforms LSTM-based adaptation for real-world transfer. It even works without domain randomization! It can handle: ✅ Uneven terrain ✅ Morphological damage ✅ Sim-to-real gaps

Excited to share our latest work: “Bio-Inspired Plastic Neural Networks for Zero-Shot Out-of-Distribution Generalization in Complex Animal-Inspired Robots” 🪲🦎 We show that Hebbian learning outperforms LSTM-based adaptation for real-world transfer. It even works without domain randomization! It can handle: ✅ Uneven terrain ✅ Morphological damage ✅ Sim-to-real gaps

Sebastian Risi

14,390 просмотров • 1 год назад

Elon just dropped a MAJOR nugget on how Tesla is going to be training Optimus to do real world tasks. They are building an Optimus Academy, which is a large scale, dedicated real-world training facility to accelerate the development of Optimus. The Academy will deploy thousands of Optimus units, potentially 10,000 to 30,000 robots, in a controlled realistic environment where they perform self-play, experiment with tasks, iterate on behaviors, and continuously generate training data through trial and error. The Tesla bots will also run millions of simulations in Tesla’s high-fidelity physics-accurate engine, allowing Optimus to close the “sim-to-real gap” by using these real-world observations to refine and validate the simulations! “You’re actually highlighting an important limitation and difference from cars. We’ll soon have 10 million cars on the road. It’s hard to duplicate that massive training flywheel. For the robot, what we’re going to need to do is build a lot of robots and put them in kind of an Optimus Academy so they can do self-play in reality. We’re actually building that out. We can have at least 10,000 Optimus robots, maybe 20-30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator, a physics-accurate reality generator, that we made for the cars. We’ll do the same thing for the robots. We actually have done that for the robots. So you have a few tens of thousands of humanoid robots doing different tasks. You can do millions of simulated robots in the simulated world. You use the tens of thousands of robots in the real world to close the simulation to reality gap. Close the sim-to-real gap.”

Elon just dropped a MAJOR nugget on how Tesla is going to be training Optimus to do real world tasks. They are building an Optimus Academy, which is a large scale, dedicated real-world training facility to accelerate the development of Optimus. The Academy will deploy thousands of Optimus units, potentially 10,000 to 30,000 robots, in a controlled realistic environment where they perform self-play, experiment with tasks, iterate on behaviors, and continuously generate training data through trial and error. The Tesla bots will also run millions of simulations in Tesla’s high-fidelity physics-accurate engine, allowing Optimus to close the “sim-to-real gap” by using these real-world observations to refine and validate the simulations! “You’re actually highlighting an important limitation and difference from cars. We’ll soon have 10 million cars on the road. It’s hard to duplicate that massive training flywheel. For the robot, what we’re going to need to do is build a lot of robots and put them in kind of an Optimus Academy so they can do self-play in reality. We’re actually building that out. We can have at least 10,000 Optimus robots, maybe 20-30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator, a physics-accurate reality generator, that we made for the cars. We’ll do the same thing for the robots. We actually have done that for the robots. So you have a few tens of thousands of humanoid robots doing different tasks. You can do millions of simulated robots in the simulated world. You use the tens of thousands of robots in the real world to close the simulation to reality gap. Close the sim-to-real gap.”

Teslaconomics

42,563 просмотров • 6 месяцев назад

We’ve all seen videos of humanoid robots performing single tasks that are very impressive, like dancing or karate. But training humanoid robots to perform a wide range of complex motions is difficult. GMT is a general-purpose policy which can learn a wide range of robot motions. Watch Episode #32 of RoboPapers, with Zixuan Chen , co-hosted by Michael Cho - Rbt/Acc and Chris Paxton , to learn more.

We’ve all seen videos of humanoid robots performing single tasks that are very impressive, like dancing or karate. But training humanoid robots to perform a wide range of complex motions is difficult. GMT is a general-purpose policy which can learn a wide range of robot motions. Watch Episode #32 of RoboPapers, with Zixuan Chen , co-hosted by Michael Cho - Rbt/Acc and Chris Paxton , to learn more.

RoboPapers

16,605 просмотров • 11 месяцев назад

.Elon Musk told me he plans to build “Optimus Academy” to train an army of humanoid robots. > “millions of simulated robots in the simulated world” > “tens of thousands of robots in the real world, to close the simulation to reality gap.” Why? There’s two big differences between the way Tesla will train humanoid robots and the way it built Full Self-Driving. - FSD only requires learning three degrees of freedom: turning, accelerating, braking … and a humanoid robot has to learn how to coordinate 50+ joints. - You can sell a Tesla without FSD, which allows you to collect the millions and millions of hours of driving data needed to bootstrap your model ... but you can’t sell an Optimus that hasn’t learned how to do stuff yet. The Optimus Academy is Elon’s attempt to build the data flywheel that Tesla got for free with FSD.

.Elon Musk told me he plans to build “Optimus Academy” to train an army of humanoid robots. > “millions of simulated robots in the simulated world” > “tens of thousands of robots in the real world, to close the simulation to reality gap.” Why? There’s two big differences between the way Tesla will train humanoid robots and the way it built Full Self-Driving. - FSD only requires learning three degrees of freedom: turning, accelerating, braking … and a humanoid robot has to learn how to coordinate 50+ joints. - You can sell a Tesla without FSD, which allows you to collect the millions and millions of hours of driving data needed to bootstrap your model ... but you can’t sell an Optimus that hasn’t learned how to do stuff yet. The Optimus Academy is Elon’s attempt to build the data flywheel that Tesla got for free with FSD.

Dwarkesh Patel

551,640 просмотров • 6 месяцев назад

Teaching robots to perform dexterous manipulation tasks currently requires teleoperation, which limits demonstration quality, speed, and scalability. Instead, why not use human videos? The problem is that a human hand isn’t a robot hand, so data must be retargeted using simulation to resolve issues like collisions and interpenetration when controlling the hand. In VideoManip, Hongyi Chen and co-authors built a system to solve this problem, taking in RGB videos of humans performing manipulation tasks and using them to create accurate simulations with which to learn robot policies. Watch episode #73 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton, now to learn more!

Teaching robots to perform dexterous manipulation tasks currently requires teleoperation, which limits demonstration quality, speed, and scalability. Instead, why not use human videos? The problem is that a human hand isn’t a robot hand, so data must be retargeted using simulation to resolve issues like collisions and interpenetration when controlling the hand. In VideoManip, Hongyi Chen and co-authors built a system to solve this problem, taking in RGB videos of humans performing manipulation tasks and using them to create accurate simulations with which to learn robot policies. Watch episode #73 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton, now to learn more!

RoboPapers

27,360 просмотров • 4 месяцев назад

🚨🇺🇸FIGURE ROBOT LEARNS TO WALK IN HOURS Turns out you can teach a robot to walk—if you’ve got a few years to burn in a physics simulator (or a few hours of GPU time). Figure’s team trained a neural network using reinforcement learning to mimic natural human locomotion, and the wild part is it worked on the first try with real hardware. No tuning. No fiddling. Just plug and strut. Thanks to sim-to-real transfer and some sneaky domain randomization, the robot didn’t just wobble out of the lab—it walked like it had somewhere to be. Next up: running. Not terrifying at all. Source: Figure

🚨🇺🇸FIGURE ROBOT LEARNS TO WALK IN HOURS Turns out you can teach a robot to walk—if you’ve got a few years to burn in a physics simulator (or a few hours of GPU time). Figure’s team trained a neural network using reinforcement learning to mimic natural human locomotion, and the wild part is it worked on the first try with real hardware. No tuning. No fiddling. Just plug and strut. Thanks to sim-to-real transfer and some sneaky domain randomization, the robot didn’t just wobble out of the lab—it walked like it had somewhere to be. Next up: running. Not terrifying at all. Source: Figure

Mario Nawfal

117,866 просмотров • 1 год назад

Researchers have developed Humanoid-GPT, a new AI system that helps humanoid robots perform complex movements and tasks. Tested on the Unitree G1 robot, it enables real-time whole-body control and can perform new actions without special training. During demonstrations, the robot successfully carried out a variety of movements and tasks, showing strong potential for real-world applications.

Researchers have developed Humanoid-GPT, a new AI system that helps humanoid robots perform complex movements and tasks. Tested on the Unitree G1 robot, it enables real-time whole-body control and can perform new actions without special training. During demonstrations, the robot successfully carried out a variety of movements and tasks, showing strong potential for real-world applications.

Space and Technology

22,234 просмотров • 2 месяцев назад

“Dexterous capability is directly proportional to the size of the addressable market for general-purpose humanoid robots,” says James Wells, interim CEO and CCO. Sanctuary AI announces its latest technical milestone using hydraulic actuation, opening up a wide range of industrial and high value work tasks that require fine manipulation. #AI #Robotics

“Dexterous capability is directly proportional to the size of the addressable market for general-purpose humanoid robots,” says James Wells, interim CEO and CCO. Sanctuary AI announces its latest technical milestone using hydraulic actuation, opening up a wide range of industrial and high value work tasks that require fine manipulation. #AI #Robotics

Sanctuary AI

16,583 просмотров • 1 год назад

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

Physical Intelligence

440,982 просмотров • 5 месяцев назад

Super excited to announce SIMA 2! It’s a general agent that can understand & reason about complex instructions and complete tasks in simulated game worlds, even ones it has never seen before. Incredible to see how it can learn just from self-play… a crucial step towards AGI

Super excited to announce SIMA 2! It’s a general agent that can understand & reason about complex instructions and complete tasks in simulated game worlds, even ones it has never seen before. Incredible to see how it can learn just from self-play… a crucial step towards AGI

Demis Hassabis

154,275 просмотров • 9 месяцев назад

Trained a humanoid entirely in a 3D scan of the office. Zero real-world fine-tuning. It just walked in and worked. RL needs hundreds of thousands of attempts, and real robots can't afford to crash. A misjudged gap or a glass door collision breaks hardware and costs hours resetting. So you train in a sim. But sim policies usually train on randomized, untextured geometry; depth is easy to fake. The robot learns structure, not the real world: no materials, no lighting, no idea what anything actually is. RGB cameras carry all of that but training RGB policies in generic fake worlds won’t generalize to the real world. Niantic Spatial 🌎 Scaniverse reconstructs your scan of the real deployment site. One 360° camera walkthrough → photorealistic 3D Gaussian splat at metric scale → collision mesh pulled from the same reconstruction, so vision and physics match exactly. Drops straight into NVIDIA Isaac Sim/Lab, no manual conversion. Flexion simulation-first approach then seamlessly enables the training of RGB-only nav policies inside that reconstruction. With added domain randomization + large image encoders for robustness, this deploys straight to hardware. No real-world fine-tuning. Deployment: months of on-site adaptation → days. Tune into the NVIDIA livestream on 12 August to hear how these companies are closing the sim2real gap: NVIDIA Robotics ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Trained a humanoid entirely in a 3D scan of the office. Zero real-world fine-tuning. It just walked in and worked. RL needs hundreds of thousands of attempts, and real robots can't afford to crash. A misjudged gap or a glass door collision breaks hardware and costs hours resetting. So you train in a sim. But sim policies usually train on randomized, untextured geometry; depth is easy to fake. The robot learns structure, not the real world: no materials, no lighting, no idea what anything actually is. RGB cameras carry all of that but training RGB policies in generic fake worlds won’t generalize to the real world. Niantic Spatial 🌎 Scaniverse reconstructs your scan of the real deployment site. One 360° camera walkthrough → photorealistic 3D Gaussian splat at metric scale → collision mesh pulled from the same reconstruction, so vision and physics match exactly. Drops straight into NVIDIA Isaac Sim/Lab, no manual conversion. Flexion simulation-first approach then seamlessly enables the training of RGB-only nav policies inside that reconstruction. With added domain randomization + large image encoders for robustness, this deploys straight to hardware. No real-world fine-tuning. Deployment: months of on-site adaptation → days. Tune into the NVIDIA livestream on 12 August to hear how these companies are closing the sim2real gap: NVIDIA Robotics ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

119,091 просмотров • 20 дней назад

Elon Musk in new interview on how Tesla’s Optimus robot will learn more quickly: "What we're going to need to do is build a lot of robots and put them in a kind of like a "Optimus Academy" so they can do self-play in reality. We're actually building that out. We're gonna have at least 10,000 Optimus robots, maybe 20,000 or 30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator. A physics-accurate reality generator that we made for the cars. We'll do the same things for the robots, and actually have done that for the robots. You can do millions of simulated robots in a simulated world, and you use the tens of thousands in the real world to close the simulation to reality gap.”

Elon Musk in new interview on how Tesla’s Optimus robot will learn more quickly: "What we're going to need to do is build a lot of robots and put them in a kind of like a "Optimus Academy" so they can do self-play in reality. We're actually building that out. We're gonna have at least 10,000 Optimus robots, maybe 20,000 or 30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator. A physics-accurate reality generator that we made for the cars. We'll do the same things for the robots, and actually have done that for the robots. You can do millions of simulated robots in a simulated world, and you use the tens of thousands in the real world to close the simulation to reality gap.”

Sawyer Merritt

250,956 просмотров • 6 месяцев назад

ELON: TESLA’S ROBOTS ARE MOVING FROM SIMULATIONS TO REALITY “For the robot, we’re going to need to build a lot of them and put them in an Optimus Academy so they can do self-play in reality and test different tasks. Tesla has quite a good, physics accurate, reality-generator, and we’re doing that for the robots. So you can do millions of robots in a simulated world, and then 10k in the real world to close the simulation to reality gap. Grok would orchestrate the behavior of the Optimus robots.” Source: Cheeky Pint Podcast, Elon Musk

ELON: TESLA’S ROBOTS ARE MOVING FROM SIMULATIONS TO REALITY “For the robot, we’re going to need to build a lot of them and put them in an Optimus Academy so they can do self-play in reality and test different tasks. Tesla has quite a good, physics accurate, reality-generator, and we’re doing that for the robots. So you can do millions of robots in a simulated world, and then 10k in the real world to close the simulation to reality gap. Grok would orchestrate the behavior of the Optimus robots.” Source: Cheeky Pint Podcast, Elon Musk

Mario Nawfal

19,289 просмотров • 6 месяцев назад

Whether teleoperated or autonomous, whole body movements like picking objects from the ground are challenging for humanoid robots. A UC San Diego paper introduces a framework that combines sim-to-real RL and trajectory optimization for adaptive, real-time whole body control.

Whether teleoperated or autonomous, whole body movements like picking objects from the ground are challenging for humanoid robots. A UC San Diego paper introduces a framework that combines sim-to-real RL and trajectory optimization for adaptive, real-time whole body control.

The Humanoid Hub

27,879 просмотров • 1 год назад