Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Turning video into humanoid robot motion! 🤳🏼 Training humanoid robots needs huge amounts of motion data, but real-world capture doesn’t scale. Mocap is expensive, dangerous edge cases are rare, and you can’t ask humans to repeatedly fall or crash. Video2Robot tackles this by converting videos into physics-grounded humanoid simulations.... Motion is generated to respect balance, inertia, ground contact, and joint limits, then directly retargeted to robot simulators. One prompt can generate a full humanoid motion sequence, including multi-agent interactions and failure cases like falls or collisions, scenarios that are hard or impossible to capture safely in the real world. The pipeline is model-agnostic and works with existing video generators, making it a practical way to scale data for robots. If robots are going to operate in the real world, they need to be trained on the failures too, not just the perfect demos. Here's the GitHub: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →show more

Lukas Ziegler

64,338 subscribers

60,070 views • 10 months ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

15 Comments

AIM Intelligence10 months ago

Happy to see our work getting recognition 🙌🏻

Lukas Ziegler10 months ago

@AIM_Intel You rock 🎸

David Pantera10 months ago

This is awesome! The product applications for remote operation are huge if this is fast enough. Is the inference time low enough to use this for real-time teleoperation, or is it strictly for offline training?

Abaka AI10 months ago

Does this mean the Best Fails of 2025 Try Not To Laugh Challenge YouTube compilations are now technically a high-value training dataset? Finally, people tripping on camera is critical safety data for AGI. What a time to be alive!

grasgor10 months ago

Wait so I can take clips of messi dribbling and make a robot dribble like him?

Camilla Mazzoleni10 months ago

how much is this demo hardcoded? 🙃

Robot ID10 months ago

Dope 🔥

Chuck Petras10 months ago

@BrianRoemmele

SimTooReal1 month ago

the robot on camera is not the robot in the loss. if limits/inertias are soft, you trained a fiction.

Himanshu Kumar10 months ago

Video-based motion capture achieved 78% accuracy in replicating human movements for robots.

Ye Olde Pinball Shoppe10 months ago

Oh that is sick. Congrats to the developers!

Connor10 months ago

Umm, literally playing a free to play video game from China that does this for custom emote creation.....Pretty sure this isnt new or industry changing.

Don Stein10 months ago

So sick Lukas

stone🛞10 months ago

This is cracked and im here for it

Alexandra Levy10 months ago

This is how we scale robot learning! So cool.

Related Videos

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 views • 1 year ago

Full-body joint tracking + smart gloves capture every nuance of human motion, which is retargeted in real-time to a humanoid robot (here, Tesla Optimus) for smooth teleoperation. This is the power of inertial motion capture. More than control, mocap is the gold standard for collecting high-fidelity humanoid training data. Advanced AI algorithms transform raw accelerometer streams into precise 3D joint poses, turning human movement into lifelike robot action.

Full-body joint tracking + smart gloves capture every nuance of human motion, which is retargeted in real-time to a humanoid robot (here, Tesla Optimus) for smooth teleoperation. This is the power of inertial motion capture. More than control, mocap is the gold standard for collecting high-fidelity humanoid training data. Advanced AI algorithms transform raw accelerometer streams into precise 3D joint poses, turning human movement into lifelike robot action.

Reborn

95,605 views • 1 year ago

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 views • 1 year ago

German robotics company NEURA Robotics has created NEURA Gym: a large-scale, physical AI gym and training ground where hundreds of robots, including the humanoid 4NE-1, learn through real-world interactions. The company is collaborating with the Neuraverse ecosystem to create the world's largest repository of physical training data, promoting the application of humanoid robots.

German robotics company NEURA Robotics has created NEURA Gym: a large-scale, physical AI gym and training ground where hundreds of robots, including the humanoid 4NE-1, learn through real-world interactions. The company is collaborating with the Neuraverse ecosystem to create the world's largest repository of physical training data, promoting the application of humanoid robots.

CyberRobo

73,731 views • 1 year ago

A planetary roller screw looks simple, but it’s one of the key enablers for humanoid robots. It converts rotational motion into precise linear motion with high efficiency, high load capacity, and extremely smooth control. In humanoid robots, this is exactly what you need for: joint actuation linear actuators in arms/legs high-force, high-precision movement under continuous cycles Compared to ball screws, it offers better durability and load distribution—critical for robots that must operate 24/7 in the real world. Small component, huge impact. This is the kind of “hidden hardware” that makes humanoid robotics actually work.

A planetary roller screw looks simple, but it’s one of the key enablers for humanoid robots. It converts rotational motion into precise linear motion with high efficiency, high load capacity, and extremely smooth control. In humanoid robots, this is exactly what you need for: joint actuation linear actuators in arms/legs high-force, high-precision movement under continuous cycles Compared to ball screws, it offers better durability and load distribution—critical for robots that must operate 24/7 in the real world. Small component, huge impact. This is the kind of “hidden hardware” that makes humanoid robotics actually work.

Steven Cheng

69,970 views • 3 months ago

A cute humanoid robot from NYC Fauna Robotics has released a humanoid robot called Sprout, which is approximately 1 meter tall, lightweight, quiet, and features no dangerous edges and soft skin. Its appearance and interactions are designed to be "approachable," like a social companion. It is currently offered as a general-purpose robotics platform for developers, providing basic movement, perception, navigation, and expression capabilities, allowing real-world use cases to emerge within the community. Many humanoid robots are born in factories and warehouses, but Fauna Robotics aims to bring humanoid robots back into human living spaces. Fauna believes that if humanoid robots are to become as ubiquitous as computers and mobile phones in the future, the starting point must be a safe, friendly form suitable for homes, schools, and offices. Sprout is their first step in this direction.

A cute humanoid robot from NYC Fauna Robotics has released a humanoid robot called Sprout, which is approximately 1 meter tall, lightweight, quiet, and features no dangerous edges and soft skin. Its appearance and interactions are designed to be "approachable," like a social companion. It is currently offered as a general-purpose robotics platform for developers, providing basic movement, perception, navigation, and expression capabilities, allowing real-world use cases to emerge within the community. Many humanoid robots are born in factories and warehouses, but Fauna Robotics aims to bring humanoid robots back into human living spaces. Fauna believes that if humanoid robots are to become as ubiquitous as computers and mobile phones in the future, the starting point must be a safe, friendly form suitable for homes, schools, and offices. Sprout is their first step in this direction.

CyberRobo

19,516 views • 9 months ago

Training humanoid robots on teleoperation alone won’t scale, says Rhoda AI CEO Jagdeep Singh. Unlike self-driving cars - which basically have four actuators (left, right, speed up, and slow down) and operate in a single environment (the road) - humanoid robots are completely different: "You're dealing with the full dexterity of a human hand - 20 degrees of freedom per hand. Every object is different. Every type of task is different." The problem isn't just the quantity of data - the bigger issue is diversity in the data, and it's why many humanoid robot demos will struggle to adapt to the real world: "If all the data you have is data that you've intentionally collected, then you almost by definition haven't seen the corner cases. You haven't seen all those edge scenarios that cause failure." Jagdeep says teleoperation is useful for fine-tuning robot behavior, but for pretraining, it's completely inadequate.

Training humanoid robots on teleoperation alone won’t scale, says Rhoda AI CEO Jagdeep Singh. Unlike self-driving cars - which basically have four actuators (left, right, speed up, and slow down) and operate in a single environment (the road) - humanoid robots are completely different: "You're dealing with the full dexterity of a human hand - 20 degrees of freedom per hand. Every object is different. Every type of task is different." The problem isn't just the quantity of data - the bigger issue is diversity in the data, and it's why many humanoid robot demos will struggle to adapt to the real world: "If all the data you have is data that you've intentionally collected, then you almost by definition haven't seen the corner cases. You haven't seen all those edge scenarios that cause failure." Jagdeep says teleoperation is useful for fine-tuning robot behavior, but for pretraining, it's completely inadequate.

TBPN

19,218 views • 7 months ago

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

50,130 views • 10 months ago

Jensen: The humanoid robot is likely the only robot that will work – because technology needs scale, and most robots we've had so far are too low volume to drive the flywheel of technology improvements. The humanoid robot is likely to be the next multi-trillion-dollar industry.

Jensen: The humanoid robot is likely the only robot that will work – because technology needs scale, and most robots we've had so far are too low volume to drive the flywheel of technology improvements. The humanoid robot is likely to be the next multi-trillion-dollar industry.

The Humanoid Hub

84,720 views • 1 year ago

Yann LeCun says today's humanoid robot demos look impressive, but the robots are very stupid in the real world "the missing piece is not hardware; it is AI that can't reason, plan, or adapt like humans" Robot companies are betting AI will make them smart enough to sell at scale within 3-5 years It's a big bet

Yann LeCun says today's humanoid robot demos look impressive, but the robots are very stupid in the real world "the missing piece is not hardware; it is AI that can't reason, plan, or adapt like humans" Robot companies are betting AI will make them smart enough to sell at scale within 3-5 years It's a big bet

Haider.

37,274 views • 7 months ago

Two wheeled humanoid robots are being deployed at JD's 7FRESH supermarket in Taiyuan… They are assisting with restocking, shelving, and providing product tasting and guidance. (VR TeleOP) From Beijing to Tianjin and Taiyuan…JD is introducing humanoid robot employees to its retail supermarkets in large numbers to enhance the shopping experience and alleviate peak customer traffic. But now,more importantly,TeleOP data These humanoid robots are from Beijing-based DEXTELEOP's TA2… who have also entered into a strategic partnership with JD. They are being deployed through JD's nationwide retail channels(20k). Placing data collection in real-world operational scenarios and achieving commercial success is a significant achievement. The physical world is the largest data collection center for humanoid robots.

Two wheeled humanoid robots are being deployed at JD's 7FRESH supermarket in Taiyuan… They are assisting with restocking, shelving, and providing product tasting and guidance. (VR TeleOP) From Beijing to Tianjin and Taiyuan…JD is introducing humanoid robot employees to its retail supermarkets in large numbers to enhance the shopping experience and alleviate peak customer traffic. But now,more importantly,TeleOP data These humanoid robots are from Beijing-based DEXTELEOP's TA2… who have also entered into a strategic partnership with JD. They are being deployed through JD's nationwide retail channels(20k). Placing data collection in real-world operational scenarios and achieving commercial success is a significant achievement. The physical world is the largest data collection center for humanoid robots.

CyberRobo

12,941 views • 2 months ago

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

21,042 views • 8 months ago

Jim Fan says NVIDIA trained humanoid robots to walk and move like humans -- zero-shot transfer from simulation to the real world. 10 years of learning, compressed into 2 hours of simulation. Turns out you don’t need a giant model to master motion. “1.5 million parameters, not billion, to capture the subconscious processing of the human body.” Big embodiment, small brain.

Jim Fan says NVIDIA trained humanoid robots to walk and move like humans -- zero-shot transfer from simulation to the real world. 10 years of learning, compressed into 2 hours of simulation. Turns out you don’t need a giant model to master motion. “1.5 million parameters, not billion, to capture the subconscious processing of the human body.” Big embodiment, small brain.

vitrupo

715,073 views • 1 year ago

🚨🚨🚨Humanoid Robots Just Stepped Into the Ring** This is not really a fight. Not yet. The robots are slow, awkward, unstable, and clearly still learning how to move like athletes. But that is exactly what makes the clip interesting. A boxing ring is one of the hardest places to fake progress. Balance matters. Reaction time matters. Footwork matters. One bad movement, and the machine is on the floor. Today, it looks funny. But every fall is data. Every awkward punch is training. And every public demo like this brings humanoid robots one step closer to moving through the world with real physical confidence. The fight is not between two robots. It is between robotics and gravity.

🚨🚨🚨Humanoid Robots Just Stepped Into the Ring** This is not really a fight. Not yet. The robots are slow, awkward, unstable, and clearly still learning how to move like athletes. But that is exactly what makes the clip interesting. A boxing ring is one of the hardest places to fake progress. Balance matters. Reaction time matters. Footwork matters. One bad movement, and the machine is on the floor. Today, it looks funny. But every fall is data. Every awkward punch is training. And every public demo like this brings humanoid robots one step closer to moving through the world with real physical confidence. The fight is not between two robots. It is between robotics and gravity.

velora

15,097 views • 2 months ago

11 years of humanoid robotics in one video. 🤖 In 2015, humanoid robots were still struggling with basic tasks like walking, climbing stairs and recovering from falls during the DARPA Robotics Challenge. Fast-forward to 2026, and humanoids are running, jumping, boxing, playing football, handling industrial tasks and operating in real-world environments. The World Humanoid Robot Games featured 2,056 robots across 51 events, including practical tasks in factories, hotels and homes. The progress is obvious. But the real question is not how impressive they look in a demo. What will humanoid robots be capable of in 2036? 👀 🎥 Media: Defense Advanced Research Projects Agency (DARPA) / Unitree Robotics ⚠️ This content is shared for informational purposes only. CTO Robotics Media is a media platform and does not own or develop the technologies shown. Credit belongs to the original creators. #HumanoidRobots #Robotics #PhysicalAI #AI #DARPA #RobotEvolution #FutureOfRobotics #EmbodiedAI #Automation #RobotTechnology #CTORoboticsMedia

11 years of humanoid robotics in one video. 🤖 In 2015, humanoid robots were still struggling with basic tasks like walking, climbing stairs and recovering from falls during the DARPA Robotics Challenge. Fast-forward to 2026, and humanoids are running, jumping, boxing, playing football, handling industrial tasks and operating in real-world environments. The World Humanoid Robot Games featured 2,056 robots across 51 events, including practical tasks in factories, hotels and homes. The progress is obvious. But the real question is not how impressive they look in a demo. What will humanoid robots be capable of in 2036? 👀 🎥 Media: Defense Advanced Research Projects Agency (DARPA) / Unitree Robotics ⚠️ This content is shared for informational purposes only. CTO Robotics Media is a media platform and does not own or develop the technologies shown. Credit belongs to the original creators. #HumanoidRobots #Robotics #PhysicalAI #AI #DARPA #RobotEvolution #FutureOfRobotics #EmbodiedAI #Automation #RobotTechnology #CTORoboticsMedia

CTO ROBOTICS Media

337,033 views • 1 month ago

The first factory in the world for the production of humanoid robots has opened in China, where "robots produce robots." Assembling one robot will take 10 minutes. It is planned to produce up to 10,000 humanoid robots annually.

The first factory in the world for the production of humanoid robots has opened in China, where "robots produce robots." Assembling one robot will take 10 minutes. It is planned to produce up to 10,000 humanoid robots annually.

𝕊𝕡𝕣𝕚𝕟𝕥𝕖𝕣 𝕻𝕣𝕖𝕤𝕤

12,407 views • 1 month ago