Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Let's reverse engineer Disney's adorable, lifelike robot! I couldn't find a whitepaper, but this is how I think it's trained: 1. The emotional behaviors are curated by Disney animation artists, keyframe by keyframe. But it cannot be "rendered" directly on the robot because it doesn't take into account the...

DrJimFan's profile picture

Jim Fan

492,333 subscribers

314,637 Aufrufe • vor 2 Jahren •via X (Twitter)

KunstBildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von Diego
Diegovor 2 Jahren

Can you recommend an affordable robot platform to play with?

Profilbild von The Rundown AI
The Rundown AIvor 1 Jahr

If you're not learning AI in 2025, you're falling behind. Join 1,000,000+ early adopters reading and learn AI in just 5 minutes a day (for free).

Profilbild von Luis C
Luis Cvor 2 Jahren

Omg its a baby AT-ST

Profilbild von Godwyll Aikins
Godwyll Aikinsvor 2 Jahren

I love your reverse engineering posts 😁

Profilbild von DesignCntrl Inc.
DesignCntrl Inc.vor 2 Jahren

If you watch it with the sound off its just a robot kiwi bird.

Profilbild von Weather Report
Weather Reportvor 2 Jahren

"This is how I think it's done" is super valuable coming from you Thank you Dr Jim Fan♥️ Please make it a series!

Profilbild von Jana
Janavor 2 Jahren

Whatever the wizardry, I luvz himz.

Profilbild von airleacher
airleachervor 2 Jahren

This is the coolest thing I have ever seen

Profilbild von Berket
Berketvor 2 Jahren

Wow, these these micro movements of the robot definitely gives the robot more personality. Like the little antenna bopping around when it moves its head or slides down

Profilbild von Bin ZHOU
Bin ZHOUvor 2 Jahren

Why not read their paper directly:

Profilbild von irow
irowvor 2 Jahren

It’s BD-1! (Just thicker)

Ähnliche Videos

Let's reverse engineer this demo. You need 3 things: (1) robust hardware and motor designs that treat simulation as first-class citizen; (2) a human motion capture ("mocap") dataset, such as those for film and gaming characters; (3) massively parallel RL training in GPU-accelerated simulation. Last October, our team trained a 1.5M parameter foundation model called HOVER for such agile motor control. It follows this recipe, roughly speaking (details in thread): (1) Simulation used to be an after-thought. Now, it has to be part of the hardware design process. If your robot doesn't simulate well, you can kiss RL goodbye. Hardware-simulation co-design is a very interesting emergent topic that only becomes meaningful with today's compute capability. (2) Human mocap dataset to produce natural-looking walking and running gaits. That's one huge advantage of using humanoid robot - you get to imitate from tons of human motions that were originally captured for movies or AAA games. At least 3 ways to use the data: - For initialization: pre-train the neural net to imitate human, and then finetune it into the robot form factor with physics turned on; - For reward function: penalize any deviations from the target pose; - For representation learning: treat the human poses as a "motion prior" to constrain the space of robot behaviors. (3) Shove the above into Isaac sim, add a lot of randomization, pump it through PPO, throw in a bunch of GPUs, and then watch Netflix till loss converges. If you have an urge to comment this is CGI, let me save you a few keystrokes — many academic labs now own the G1 robot in the flesh. See our team's HOVER work in the thread: 🧵
1:04

Let's reverse engineer this demo. You need 3 things: (1) robust hardware and motor designs that treat simulation as first-class citizen; (2) a human motion capture ("mocap") dataset, such as those for film and gaming characters; (3) massively parallel RL training in GPU-accelerated simulation. Last October, our team trained a 1.5M parameter foundation model called HOVER for such agile motor control. It follows this recipe, roughly speaking (details in thread): (1) Simulation used to be an after-thought. Now, it has to be part of the hardware design process. If your robot doesn't simulate well, you can kiss RL goodbye. Hardware-simulation co-design is a very interesting emergent topic that only becomes meaningful with today's compute capability. (2) Human mocap dataset to produce natural-looking walking and running gaits. That's one huge advantage of using humanoid robot - you get to imitate from tons of human motions that were originally captured for movies or AAA games. At least 3 ways to use the data: - For initialization: pre-train the neural net to imitate human, and then finetune it into the robot form factor with physics turned on; - For reward function: penalize any deviations from the target pose; - For representation learning: treat the human poses as a "motion prior" to constrain the space of robot behaviors. (3) Shove the above into Isaac sim, add a lot of randomization, pump it through PPO, throw in a bunch of GPUs, and then watch Netflix till loss converges. If you have an urge to comment this is CGI, let me save you a few keystrokes — many academic labs now own the G1 robot in the flesh. See our team's HOVER work in the thread: 🧵

Jim Fan

216,139 Aufrufe • vor 1 Jahr

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
53:45

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,334 Aufrufe • vor 1 Monat

Foundation models are enough to solve robotics! Unfortunately, this is not true. We keep hearing that Vision-Language-Action (VLA) models struggle because of the gap between static training and the dynamic real world. A German startup (Sereact) just released a solution that bridges this gap perfectly. They are introducing a new paradigm called Interactive RL Policy Patching. It's a distributed framework that allows robots to self-learn from human corrections without needing full retraining. When a robot fails, a human operator provides a brief "patch" or demonstration. The system then uses online off-policy reinforcement learning to update the behavior instantly. This is powered by a massive foundation model trained on hundreds of millions of interactions from over 100 deployed robot stations. The best part is the distributed parameter synchronization... When one robot learns a fix, the update is published fleet-wide... meaning the entire swarm gets smarter from a single human intervention. They are already proving this on complex manipulation tasks like shoe unboxing and screw sorting, drastically reducing the data needed to handle edge cases. Real-world environments are unforgiving, and I love seeing systems that can actually adapt on the fly! 📍 More info:
0:38

Foundation models are enough to solve robotics! Unfortunately, this is not true. We keep hearing that Vision-Language-Action (VLA) models struggle because of the gap between static training and the dynamic real world. A German startup (Sereact) just released a solution that bridges this gap perfectly. They are introducing a new paradigm called Interactive RL Policy Patching. It's a distributed framework that allows robots to self-learn from human corrections without needing full retraining. When a robot fails, a human operator provides a brief "patch" or demonstration. The system then uses online off-policy reinforcement learning to update the behavior instantly. This is powered by a massive foundation model trained on hundreds of millions of interactions from over 100 deployed robot stations. The best part is the distributed parameter synchronization... When one robot learns a fix, the update is published fleet-wide... meaning the entire swarm gets smarter from a single human intervention. They are already proving this on complex manipulation tasks like shoe unboxing and screw sorting, drastically reducing the data needed to handle edge cases. Real-world environments are unforgiving, and I love seeing systems that can actually adapt on the fly! 📍 More info:

Ilir Aliu

18,210 Aufrufe • vor 5 Monaten

NAVIAI-I3, a more robust humanoid robot, was launched by the Zhejiang Humanoid Robotics Innovation Center. Standing 1.8 meters tall, it can run like a human and walk on various terrains. This humanoid robot appears to be primarily designed for specialized industrial tasks, such as security patrols in underground parking lots. They previously launched the lightweight bipedal NVAI-2 and a wheeled humanoid robot. The bipedal robot is used in manufacturing operations, while the wheeled humanoid robot is used for unstructured, flexible tasks such as warehouse sorting, supermarket shelving, and office cleaning. We filmed this demonstration at the World Robot Conference; you can see it in the comments section.
1:00

NAVIAI-I3, a more robust humanoid robot, was launched by the Zhejiang Humanoid Robotics Innovation Center. Standing 1.8 meters tall, it can run like a human and walk on various terrains. This humanoid robot appears to be primarily designed for specialized industrial tasks, such as security patrols in underground parking lots. They previously launched the lightweight bipedal NVAI-2 and a wheeled humanoid robot. The bipedal robot is used in manufacturing operations, while the wheeled humanoid robot is used for unstructured, flexible tasks such as warehouse sorting, supermarket shelving, and office cleaning. We filmed this demonstration at the World Robot Conference; you can see it in the comments section.

CyberRobo

15,521 Aufrufe • vor 8 Monaten

Can GPT-4 teach a robot hand to do pen spinning tricks better than you do? I'm excited to announce Eureka, an open-ended agent that designs reward functions for robot dexterity at super-human level. It’s like Voyager in the space of a physics simulator API! Eureka bridges the gap between high-level reasoning (coding) and low-level motor control. It is a “hybrid-gradient architecture”: a black box, inference-only LLM instructs a white box, learnable neural network. The outer loop runs GPT-4 to refine the reward function (gradient-free), while the inner loop runs reinforcement learning to train a robot controller (gradient-based). We are able to scale up Eureka thanks to IsaacGym, a GPU-accelerated physics simulator that speeds up reality by 1000x. On a benchmark suite of 29 tasks across 10 robots, Eureka rewards outperform expert human-written ones on 83% of the tasks by 52% improvement margin on average. We are surprised that Eureka is able to learn pen spinning tricks, which are very difficult even for CGI artists to animate frame by frame! Eureka also enables a new form of in-context RLHF, which is able to incorporate a human operator’s feedback in natural language to steer and align the reward functions. It can serve as a powerful co-pilot for robot engineers to design sophisticated motor behaviors. As usual, we open-source everything! Welcome you all to check out our video gallery and try the codebase today: Paper: Code: Deep dive with me: 🧵
0:24

Can GPT-4 teach a robot hand to do pen spinning tricks better than you do? I'm excited to announce Eureka, an open-ended agent that designs reward functions for robot dexterity at super-human level. It’s like Voyager in the space of a physics simulator API! Eureka bridges the gap between high-level reasoning (coding) and low-level motor control. It is a “hybrid-gradient architecture”: a black box, inference-only LLM instructs a white box, learnable neural network. The outer loop runs GPT-4 to refine the reward function (gradient-free), while the inner loop runs reinforcement learning to train a robot controller (gradient-based). We are able to scale up Eureka thanks to IsaacGym, a GPU-accelerated physics simulator that speeds up reality by 1000x. On a benchmark suite of 29 tasks across 10 robots, Eureka rewards outperform expert human-written ones on 83% of the tasks by 52% improvement margin on average. We are surprised that Eureka is able to learn pen spinning tricks, which are very difficult even for CGI artists to animate frame by frame! Eureka also enables a new form of in-context RLHF, which is able to incorporate a human operator’s feedback in natural language to steer and align the reward functions. It can serve as a powerful co-pilot for robot engineers to design sophisticated motor behaviors. As usual, we open-source everything! Welcome you all to check out our video gallery and try the codebase today: Paper: Code: Deep dive with me: 🧵

Jim Fan

2,673,990 Aufrufe • vor 2 Jahren

Karpathy's prediction about RL is coming true now! He called reward functions unreliable and argued that a single reward number is too low-dimensional to teach an agent what "good" means for complex tasks. To solve this, Agents need a knowledge-guided review as a higher-dimensional feedback channel. Every major AI lab trains models with RL today (OpenAI, Anthropic, DeepSeek). And their key bottleneck has always been the reward functions. GRPO by DeepSeek worked well for math and code because the environment gave a binary signal. But for real agent tasks, someone still has to hand-code the scoring function. That takes days and breaks every time the pipeline changes. RULER (implemented in OpenPipe ART, 10k stars) addresses the exact problem Karpathy identified. The reward criteria are defined in plain English, and an LLM evaluates each trajectory against that description to provide feedback for training. I trained a Qwen3 1.4B agent that plays 2048 using GRPO with this exact workflow. In this case, the agent saw the board, picked a direction, and RULER evaluated the outcome, all from this natural language definition. You can see the full implementation on GitHub and try it yourself. Here's the ART Repo: (don't forget to star it ⭐ ) Just like RLHF replaced manual rankings and GRPO replaced the critic model, natural language rewards are replacing hand-coded scoring functions. RL reward engineering is now prompt engineering. I wrote a full walkthrough covering RL for LLM agents, from RLHF to GRPO to RULER, in the article below.
0:25

Karpathy's prediction about RL is coming true now! He called reward functions unreliable and argued that a single reward number is too low-dimensional to teach an agent what "good" means for complex tasks. To solve this, Agents need a knowledge-guided review as a higher-dimensional feedback channel. Every major AI lab trains models with RL today (OpenAI, Anthropic, DeepSeek). And their key bottleneck has always been the reward functions. GRPO by DeepSeek worked well for math and code because the environment gave a binary signal. But for real agent tasks, someone still has to hand-code the scoring function. That takes days and breaks every time the pipeline changes. RULER (implemented in OpenPipe ART, 10k stars) addresses the exact problem Karpathy identified. The reward criteria are defined in plain English, and an LLM evaluates each trajectory against that description to provide feedback for training. I trained a Qwen3 1.4B agent that plays 2048 using GRPO with this exact workflow. In this case, the agent saw the board, picked a direction, and RULER evaluated the outcome, all from this natural language definition. You can see the full implementation on GitHub and try it yourself. Here's the ART Repo: (don't forget to star it ⭐ ) Just like RLHF replaced manual rankings and GRPO replaced the critic model, natural language rewards are replacing hand-coded scoring functions. RL reward engineering is now prompt engineering. I wrote a full walkthrough covering RL for LLM agents, from RLHF to GRPO to RULER, in the article below.

Avi Chawla

349,743 Aufrufe • vor 1 Monat

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.
0:05

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

Rohan Paul

37,121 Aufrufe • vor 4 Monaten

We trained a robot dog to balance and walk on top of a yoga ball purely in simulation, and then transfer zero-shot to the real world. No fine-tuning. Just works. I’m excited to announce DrEureka, an LLM agent that writes code to train robot skills in simulation, and writes more code to bridge the difficult simulation-reality gap. It fully automates the pipeline from new skill learning to real-world deployment. The Yoga ball task is particularly hard because it is not possible to accurately simulate the bouncy ball surface. Yet DrEureka has no trouble searching over a vast space of sim-to-real configurations, and enables the dog to steer the ball on various terrains, even walking sideways! Traditionally, the sim-to-real transfer is achieved by domain randomization, a tedious process that requires expert human roboticists to stare at every parameter and adjust by hand. Frontier LLMs like GPT-4 have tons of built-in physical intuition for friction, damping, stiffness, gravity, etc. We are (mildly) surprised to find that DrEureka can tune these parameters competently and explain its reasoning well. DrEureka builds on our prior work Eureka, the algorithm that teaches a 5-finger robot hand to do pen spinning. It takes one step further on our quest to automate the entire robot learning pipeline by an AI agent system. One model that outputs strings will supervise another model that outputs torque control. We open-source everything! Welcome you all to check out the paper, more videos, and try the codebase today: Code:
0:24

We trained a robot dog to balance and walk on top of a yoga ball purely in simulation, and then transfer zero-shot to the real world. No fine-tuning. Just works. I’m excited to announce DrEureka, an LLM agent that writes code to train robot skills in simulation, and writes more code to bridge the difficult simulation-reality gap. It fully automates the pipeline from new skill learning to real-world deployment. The Yoga ball task is particularly hard because it is not possible to accurately simulate the bouncy ball surface. Yet DrEureka has no trouble searching over a vast space of sim-to-real configurations, and enables the dog to steer the ball on various terrains, even walking sideways! Traditionally, the sim-to-real transfer is achieved by domain randomization, a tedious process that requires expert human roboticists to stare at every parameter and adjust by hand. Frontier LLMs like GPT-4 have tons of built-in physical intuition for friction, damping, stiffness, gravity, etc. We are (mildly) surprised to find that DrEureka can tune these parameters competently and explain its reasoning well. DrEureka builds on our prior work Eureka, the algorithm that teaches a 5-finger robot hand to do pen spinning. It takes one step further on our quest to automate the entire robot learning pipeline by an AI agent system. One model that outputs strings will supervise another model that outputs torque control. We open-source everything! Welcome you all to check out the paper, more videos, and try the codebase today: Code:

Jim Fan

908,690 Aufrufe • vor 2 Jahren

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data. 2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the *motion* of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro -> RoboCasa produces N (varying visuals) -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are building tools to enable everyone in the ecosystem to scale up with us. Links in thread:
1:21

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data. 2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the *motion* of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro -> RoboCasa produces N (varying visuals) -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are building tools to enable everyone in the ecosystem to scale up with us. Links in thread:

Jim Fan

364,380 Aufrufe • vor 1 Jahr

Tesla's Chief Designer Franz on Optimus: "It's going to be huge piece of Tesla, a huge piece in our world, as we look into the future, the ability to take menial tasks and things that you don't necessarily want to do and have them uploaded to somebody, a human, that can do that. We've been designing the robot that really looks at how humans work in the world. This world is designed around humans, and so I think having a robot that can do the things that humans can do can really start to take those menial tasks away and enrich in our lives, make our lives better we get to spend more time doing the things that we love. It's been incredibly fast paced, but the progress is, like, unbelievable in the short amount of time that the team has been working on it, and we've been growing the team and building the project the same time. I think we're breaking ground in this space everyday. The team is really energized and fired out. Elon Musk is incredibly involved in this. It's a great project to be a part of, and seeing where the future is gonna go and how it's going to enrich our lives is super exciting."
1:13

Tesla's Chief Designer Franz on Optimus: "It's going to be huge piece of Tesla, a huge piece in our world, as we look into the future, the ability to take menial tasks and things that you don't necessarily want to do and have them uploaded to somebody, a human, that can do that. We've been designing the robot that really looks at how humans work in the world. This world is designed around humans, and so I think having a robot that can do the things that humans can do can really start to take those menial tasks away and enrich in our lives, make our lives better we get to spend more time doing the things that we love. It's been incredibly fast paced, but the progress is, like, unbelievable in the short amount of time that the team has been working on it, and we've been growing the team and building the project the same time. I think we're breaking ground in this space everyday. The team is really energized and fired out. Elon Musk is incredibly involved in this. It's a great project to be a part of, and seeing where the future is gonna go and how it's going to enrich our lives is super exciting."

DogeDesigner

202,945 Aufrufe • vor 9 Monaten

"One of the very confusing things about the models right now: how to reconcile the fact that they are doing so well on evals. And you look at the evals and you go, 'Those are pretty hard evals.' But the economic impact seems to be dramatically behind. There is [a possible] explanation. Back when people were doing pre-training, the question of what data to train on was answered, because that answer was everything. So you don't have to think if it's going to be this data or that data. When people do RL training, they say, 'Okay, we want to have this kind of RL training for this thing and that kind of RL training for that thing.' You say, 'Hey, I would love our model to do really well when we release it. I want the evals to look great. What would be RL training that could help on this task?' If you combine this with generalization of the models actually being inadequate, that has the potential to explain a lot of what we are seeing, this disconnect between eval performance and actual real-world performance"
2:21

"One of the very confusing things about the models right now: how to reconcile the fact that they are doing so well on evals. And you look at the evals and you go, 'Those are pretty hard evals.' But the economic impact seems to be dramatically behind. There is [a possible] explanation. Back when people were doing pre-training, the question of what data to train on was answered, because that answer was everything. So you don't have to think if it's going to be this data or that data. When people do RL training, they say, 'Okay, we want to have this kind of RL training for this thing and that kind of RL training for that thing.' You say, 'Hey, I would love our model to do really well when we release it. I want the evals to look great. What would be RL training that could help on this task?' If you combine this with generalization of the models actually being inadequate, that has the potential to explain a lot of what we are seeing, this disconnect between eval performance and actual real-world performance"

Dwarkesh Patel

502,060 Aufrufe • vor 7 Monaten

Cute RL paper from Disney Research for the holidays❄️ "The illusion of believability is fragile: even small inconsistencies, such as rough foot impacts or jitter, can break the character's lifelike appearance." To maximize the character's believability, they used RL to optimize for realism & acoustic stealth. They basically conditioned the policy directly on artist-created animation files, training the agent to learn Olaf's specific rolling gait from the movie rather than a standard efficient robot walk. To solve the "robotic clanking" problem, they introduced an "impact reduction" reward that penalizes high vertical velocity changes specifically at the moment of foot contact, effectively teaching the robot to "tiptoe". Happy holidays / Merry Christmas!🎄❄️
4:27

Cute RL paper from Disney Research for the holidays❄️ "The illusion of believability is fragile: even small inconsistencies, such as rough foot impacts or jitter, can break the character's lifelike appearance." To maximize the character's believability, they used RL to optimize for realism & acoustic stealth. They basically conditioned the policy directly on artist-created animation files, training the agent to learn Olaf's specific rolling gait from the movie rather than a standard efficient robot walk. To solve the "robotic clanking" problem, they introduced an "impact reduction" reward that penalizes high vertical velocity changes specifically at the moment of foot contact, effectively teaching the robot to "tiptoe". Happy holidays / Merry Christmas!🎄❄️

Chen Sun 🤖

44,010 Aufrufe • vor 6 Monaten

I've long wondered if we can make a humanoid robot do a 𝘄𝗮𝗹𝗹𝗳𝗹𝗶𝗽 - and we just made it happen by leveraging 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁 with BeyondMimic tracking! This came after our original OmniRetarget experiments, with only minor tweaks to RL training: relaxing a termination threshold and removing one reward term. The policy achieved a 𝟱/𝟱 success rate in our real-world experiments, showing the strength of high-quality, interaction-preserving motion retargeting combined with BeyondMimic’s minimal RL tracking. Here is the updated arXiv: (In Sec. V. A)
0:20

I've long wondered if we can make a humanoid robot do a 𝘄𝗮𝗹𝗹𝗳𝗹𝗶𝗽 - and we just made it happen by leveraging 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁 with BeyondMimic tracking! This came after our original OmniRetarget experiments, with only minor tweaks to RL training: relaxing a termination threshold and removing one reward term. The policy achieved a 𝟱/𝟱 success rate in our real-world experiments, showing the strength of high-quality, interaction-preserving motion retargeting combined with BeyondMimic’s minimal RL tracking. Here is the updated arXiv: (In Sec. V. A)

Zhen Wu

1,051,049 Aufrufe • vor 9 Monaten

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:
0:29

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

42,804 Aufrufe • vor 5 Monaten

Another robot from Disney! 🕺🏻 Just look at this dancing fella created by Disney Research. Creating control policies that work on real robots and handling diverse, unseen motions in physics-based character control is still difficult. The Disney team proposes a two-state solution. The first step involves learning a latent space encoding with an autoencoder. After encoding, the policy is trained to map kinematic input to dynamic output, ensuring accurate and adaptable movement. This method eliminates common mode collapse issues and improves motion encoding by separating these stages. This technique has proven effective in real robots in simulations, marking a significant leap forward in robot control. Can't wait for it in action live in 2025 :) ~~~ RT to help 1 robot find a new workplace.
0:39

Another robot from Disney! 🕺🏻 Just look at this dancing fella created by Disney Research. Creating control policies that work on real robots and handling diverse, unseen motions in physics-based character control is still difficult. The Disney team proposes a two-state solution. The first step involves learning a latent space encoding with an autoencoder. After encoding, the policy is trained to map kinematic input to dynamic output, ensuring accurate and adaptable movement. This method eliminates common mode collapse issues and improves motion encoding by separating these stages. This technique has proven effective in real robots in simulations, marking a significant leap forward in robot control. Can't wait for it in action live in 2025 :) ~~~ RT to help 1 robot find a new workplace.

Lukas Ziegler

65,887 Aufrufe • vor 1 Jahr

I made Physical AutoResearch sound simple (conceptually), but it took a village to pull off and lots of design thinking into the robot /loopcraft. The hardest part is everything we need to setup *before* pressing Enter. Here's a behind-the-scene tour: 1. Safety harness Letting 8 robots run unattended overnight means safety has to be more than a hint in the system prompt. ENPIRE hardwires it in 2 layers: (1) hard kinematic limit that trips an immediate task failure and auto-resets as soon as a robot leaves its safety envelope, and (2) a torque-limited compliant gripper so a bad contact or misaligned insertion ends in a safe stall, instead of crushing the robot or the object at hand. We make safety more conservative than usual so humans can sleep tight. In reality, we still need a few human operators to watch over the "robots of loving grace". 2. Definition of /done An agent that can edit its own reward will game it for sure. ENPIRE fixes the goalposts before the fleet can move them. Here's the recipe: Collect a few minutes of success & failure demos -> Ask agent to write code using computer vision tools to classify success and measure against groundtruth -> Agent hill-climbs on classifier until reliably good -> This classifier becomes the real-time reward function that directly computes on sensor streams -> *Freeze* the reward function before AutoResearch. It's sacred, enshrined in a Gym env that no one can touch. 3. System telemetry design Robot-seconds is by far the scarcest resource, followed by GPU-seconds, and finally tokens. We instrument all three and surface them to ENPIRE for live resource awareness rather than letting it hill-climb in a vacuum. We define: - Mean Robot Utilization ("MRU"): the fraction of wall-clock time when the robot is actively executing an experiment. Otherwise the hardware is sitting idle and waiting for the next code commit. - Mean Token Utilization ("MTU"): tokens consumed per minute, our proxy for how hard the agent is actually thinking. A low MTU means the agent is stalled, waiting on a robot rollout to finish instead of doing research. - GPU utilization: fraction of wall-clock time when GPU is active. ... and evaluate on two budget-to-outcome metrics: 1. Tokens-to-Success: token budget the fleet burns to complete /goal. 2. Time-to-Success: wall-clock time to /goal
0:42

I made Physical AutoResearch sound simple (conceptually), but it took a village to pull off and lots of design thinking into the robot /loopcraft. The hardest part is everything we need to setup *before* pressing Enter. Here's a behind-the-scene tour: 1. Safety harness Letting 8 robots run unattended overnight means safety has to be more than a hint in the system prompt. ENPIRE hardwires it in 2 layers: (1) hard kinematic limit that trips an immediate task failure and auto-resets as soon as a robot leaves its safety envelope, and (2) a torque-limited compliant gripper so a bad contact or misaligned insertion ends in a safe stall, instead of crushing the robot or the object at hand. We make safety more conservative than usual so humans can sleep tight. In reality, we still need a few human operators to watch over the "robots of loving grace". 2. Definition of /done An agent that can edit its own reward will game it for sure. ENPIRE fixes the goalposts before the fleet can move them. Here's the recipe: Collect a few minutes of success & failure demos -> Ask agent to write code using computer vision tools to classify success and measure against groundtruth -> Agent hill-climbs on classifier until reliably good -> This classifier becomes the real-time reward function that directly computes on sensor streams -> *Freeze* the reward function before AutoResearch. It's sacred, enshrined in a Gym env that no one can touch. 3. System telemetry design Robot-seconds is by far the scarcest resource, followed by GPU-seconds, and finally tokens. We instrument all three and surface them to ENPIRE for live resource awareness rather than letting it hill-climb in a vacuum. We define: - Mean Robot Utilization ("MRU"): the fraction of wall-clock time when the robot is actively executing an experiment. Otherwise the hardware is sitting idle and waiting for the next code commit. - Mean Token Utilization ("MTU"): tokens consumed per minute, our proxy for how hard the agent is actually thinking. A low MTU means the agent is stalled, waiting on a robot rollout to finish instead of doing research. - GPU utilization: fraction of wall-clock time when GPU is active. ... and evaluate on two budget-to-outcome metrics: 1. Tokens-to-Success: token budget the fleet burns to complete /goal. 2. Time-to-Success: wall-clock time to /goal

Jim Fan

100,588 Aufrufe • vor 27 Tagen

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion Robotics to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla
1:06:07

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion Robotics to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

The TWIML AI Podcast

22,264 Aufrufe • vor 6 Monaten

Dr Fei-Fei-Li explains with a simple example how everyday household chores are so extremely difficult for Robots. "If you tell a robot to open the top drawer and watch out for the vase, this is actually a really hard task for robots." because the robot must ground language into the real world. Words like "top", "drawer", and "vase" are abstract. The system has to map them to 3D locations, objects, and relations in a noisy scene. This requires robust perception, object recognition, and spatial reasoning under uncertainty. The robot also lacks human commonsense. "Watch out" implies predicting consequences, estimating clearances, and understanding that vases are fragile. Encoding such priors, like how heavy a drawer is or how a vase might tip, is very complex and difficult without rich world knowledge. Learning the behavior from rewards is tough. The success signal is very sparse here, so naive exploration almost never stumbles on a full success sequence. This makes policy learning sample inefficient and brittle, especially when the environment changes between training and deployment. A sparse reward situation is when the agent only gets a success signal at the very end, and gets little or no feedback along the way. If a robot must open a drawer without hitting a vase, it might get reward only if the drawer ends up open and the vase is intact. Every partial try before that looks the same to the learner, reward equals 0. --- From "DSAI by Dr. Osbert Tay" YT channel
0:48

Dr Fei-Fei-Li explains with a simple example how everyday household chores are so extremely difficult for Robots. "If you tell a robot to open the top drawer and watch out for the vase, this is actually a really hard task for robots." because the robot must ground language into the real world. Words like "top", "drawer", and "vase" are abstract. The system has to map them to 3D locations, objects, and relations in a noisy scene. This requires robust perception, object recognition, and spatial reasoning under uncertainty. The robot also lacks human commonsense. "Watch out" implies predicting consequences, estimating clearances, and understanding that vases are fragile. Encoding such priors, like how heavy a drawer is or how a vase might tip, is very complex and difficult without rich world knowledge. Learning the behavior from rewards is tough. The success signal is very sparse here, so naive exploration almost never stumbles on a full success sequence. This makes policy learning sample inefficient and brittle, especially when the environment changes between training and deployment. A sparse reward situation is when the agent only gets a success signal at the very end, and gets little or no feedback along the way. If a robot must open a drawer without hitting a vase, it might get reward only if the drawer ends up open and the vase is intact. Every partial try before that looks the same to the learner, reward equals 0. --- From "DSAI by Dr. Osbert Tay" YT channel

Rohan Paul

342,553 Aufrufe • vor 8 Monaten

Quadrupeds are fast. Agile. Great at locomotion. But can they manipulate? A new approach from Carnegie Mellon University, Google DeepMind, and Bosch is teaching quadrupedal robots to do more than walk, they’re learning to interact. It’s called Human2LocoMan: a system that uses human data to pretrain robot policies before finetuning on real hardware. The result? A four-legged robot that can walk, carry, organize, scoop, and sort with both single and dual-arm control. By pretraining on human motion, they cut the amount of robot data in half—while improving success rates by over 80% in unfamiliar environments. Their Modularized Cross-Embodiment Transformer (MXT) learns from both human and robot demonstrations, then generalizes those skills to physical tasks—no hardcoded behaviors required. It’s locomotion and manipulation. A quadruped that can walk and clean up after itself?
1:21

Quadrupeds are fast. Agile. Great at locomotion. But can they manipulate? A new approach from Carnegie Mellon University, Google DeepMind, and Bosch is teaching quadrupedal robots to do more than walk, they’re learning to interact. It’s called Human2LocoMan: a system that uses human data to pretrain robot policies before finetuning on real hardware. The result? A four-legged robot that can walk, carry, organize, scoop, and sort with both single and dual-arm control. By pretraining on human motion, they cut the amount of robot data in half—while improving success rates by over 80% in unfamiliar environments. Their Modularized Cross-Embodiment Transformer (MXT) learns from both human and robot demonstrations, then generalizes those skills to physical tasks—no hardcoded behaviors required. It’s locomotion and manipulation. A quadruped that can walk and clean up after itself?

Lukas Ziegler

60,546 Aufrufe • vor 1 Jahr

NVIDIA just announced EgoScale 🤖🧠 NVIDIA Research has uncovered a log-linear scaling law for robot dexterity by pretraining VLA models on over 20,000 hours of egocentric human video This massive dataset is 20 times larger than previous efforts and proves that robot intelligence follows a predictable path: the more human data, the lower the loss The secret is a simple recipe combining large-scale human pretraining with a small amount of aligned human-robot mid-training to bridge the gap In testing, this method boosted the average success rate by 54% on a 22-DoF robotic hand compared to policies built without pretraining EgoScale also enables one-shot task adaptation and works across different hardware, suggesting that human motion is a universal motor prior for robots Website: Paper: Source: NVIDIA Research #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #NVIDIA #EgoScale #GR00T
1:25

NVIDIA just announced EgoScale 🤖🧠 NVIDIA Research has uncovered a log-linear scaling law for robot dexterity by pretraining VLA models on over 20,000 hours of egocentric human video This massive dataset is 20 times larger than previous efforts and proves that robot intelligence follows a predictable path: the more human data, the lower the loss The secret is a simple recipe combining large-scale human pretraining with a small amount of aligned human-robot mid-training to bridge the gap In testing, this method boosted the average success rate by 54% on a 22-DoF robotic hand compared to policies built without pretraining EgoScale also enables one-shot task adaptation and works across different hardware, suggesting that human motion is a universal motor prior for robots Website: Paper: Source: NVIDIA Research #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #NVIDIA #EgoScale #GR00T

RoboHub🤖

43,752 Aufrufe • vor 4 Monaten