Загрузка видео...

Не удалось загрузить видео

На главную

Turning video into humanoid robot motion! 🤳🏼 Training humanoid robots needs huge amounts of motion data, but real-world capture doesn’t scale. Mocap is expensive, dangerous edge cases are rare, and you can’t ask humans to repeatedly fall or crash. Video2Robot tackles this by converting videos into physics-grounded humanoid simulations....

lukas_m_ziegler's profile picture

Lukas Ziegler

64,338 subscribers

60,070 просмотров • 10 месяцев назад •via X (Twitter)

ОбразованиеНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 15

Фото профиля AIM Intelligence
AIM Intelligence10 месяцев назад

Happy to see our work getting recognition 🙌🏻

Фото профиля Lukas Ziegler
Lukas Ziegler10 месяцев назад

@AIM_Intel You rock 🎸

Фото профиля David Pantera
David Pantera10 месяцев назад

This is awesome! The product applications for remote operation are huge if this is fast enough. Is the inference time low enough to use this for real-time teleoperation, or is it strictly for offline training?

Фото профиля Abaka AI
Abaka AI10 месяцев назад

Does this mean the Best Fails of 2025 Try Not To Laugh Challenge YouTube compilations are now technically a high-value training dataset? Finally, people tripping on camera is critical safety data for AGI. What a time to be alive!

Фото профиля grasgor
grasgor10 месяцев назад

Wait so I can take clips of messi dribbling and make a robot dribble like him?

Фото профиля Camilla Mazzoleni
Camilla Mazzoleni10 месяцев назад

how much is this demo hardcoded? 🙃

Фото профиля Robot ID
Robot ID10 месяцев назад

Dope 🔥

Фото профиля Chuck Petras
Chuck Petras10 месяцев назад

@BrianRoemmele

Фото профиля SimTooReal
SimTooReal1 месяц назад

the robot on camera is not the robot in the loss. if limits/inertias are soft, you trained a fiction.

Фото профиля Himanshu Kumar
Himanshu Kumar10 месяцев назад

Video-based motion capture achieved 78% accuracy in replicating human movements for robots.

Фото профиля Ye Olde Pinball Shoppe
Ye Olde Pinball Shoppe10 месяцев назад

Oh that is sick. Congrats to the developers!

Фото профиля Connor
Connor10 месяцев назад

Umm, literally playing a free to play video game from China that does this for custom emote creation.....Pretty sure this isnt new or industry changing.

Фото профиля Don Stein
Don Stein10 месяцев назад

So sick Lukas

Фото профиля stone🛞
stone🛞10 месяцев назад

This is cracked and im here for it

Фото профиля Alexandra Levy
Alexandra Levy10 месяцев назад

This is how we scale robot learning! So cool.

Похожие видео

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.
0:58

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 просмотров • 1 год назад

Full-body joint tracking + smart gloves capture every nuance of human motion, which is retargeted in real-time to a humanoid robot (here, Tesla Optimus) for smooth teleoperation. This is the power of inertial motion capture. More than control, mocap is the gold standard for collecting high-fidelity humanoid training data. Advanced AI algorithms transform raw accelerometer streams into precise 3D joint poses, turning human movement into lifelike robot action.
0:20

Full-body joint tracking + smart gloves capture every nuance of human motion, which is retargeted in real-time to a humanoid robot (here, Tesla Optimus) for smooth teleoperation. This is the power of inertial motion capture. More than control, mocap is the gold standard for collecting high-fidelity humanoid training data. Advanced AI algorithms transform raw accelerometer streams into precise 3D joint poses, turning human movement into lifelike robot action.

Reborn

95,605 просмотров • 1 год назад

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.
2:05

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 просмотров • 1 год назад

German robotics company NEURA Robotics has created NEURA Gym: a large-scale, physical AI gym and training ground where hundreds of robots, including the humanoid 4NE-1, learn through real-world interactions. The company is collaborating with the Neuraverse ecosystem to create the world's largest repository of physical training data, promoting the application of humanoid robots.
1:39

German robotics company NEURA Robotics has created NEURA Gym: a large-scale, physical AI gym and training ground where hundreds of robots, including the humanoid 4NE-1, learn through real-world interactions. The company is collaborating with the Neuraverse ecosystem to create the world's largest repository of physical training data, promoting the application of humanoid robots.

CyberRobo

73,731 просмотров • 1 год назад

A planetary roller screw looks simple, but it’s one of the key enablers for humanoid robots. It converts rotational motion into precise linear motion with high efficiency, high load capacity, and extremely smooth control. In humanoid robots, this is exactly what you need for: joint actuation linear actuators in arms/legs high-force, high-precision movement under continuous cycles Compared to ball screws, it offers better durability and load distribution—critical for robots that must operate 24/7 in the real world. Small component, huge impact. This is the kind of “hidden hardware” that makes humanoid robotics actually work.
0:57

A planetary roller screw looks simple, but it’s one of the key enablers for humanoid robots. It converts rotational motion into precise linear motion with high efficiency, high load capacity, and extremely smooth control. In humanoid robots, this is exactly what you need for: joint actuation linear actuators in arms/legs high-force, high-precision movement under continuous cycles Compared to ball screws, it offers better durability and load distribution—critical for robots that must operate 24/7 in the real world. Small component, huge impact. This is the kind of “hidden hardware” that makes humanoid robotics actually work.

Steven Cheng

69,970 просмотров • 3 месяцев назад

A cute humanoid robot from NYC Fauna Robotics has released a humanoid robot called Sprout, which is approximately 1 meter tall, lightweight, quiet, and features no dangerous edges and soft skin. Its appearance and interactions are designed to be "approachable," like a social companion. It is currently offered as a general-purpose robotics platform for developers, providing basic movement, perception, navigation, and expression capabilities, allowing real-world use cases to emerge within the community. Many humanoid robots are born in factories and warehouses, but Fauna Robotics aims to bring humanoid robots back into human living spaces. Fauna believes that if humanoid robots are to become as ubiquitous as computers and mobile phones in the future, the starting point must be a safe, friendly form suitable for homes, schools, and offices. Sprout is their first step in this direction.
1:01

A cute humanoid robot from NYC Fauna Robotics has released a humanoid robot called Sprout, which is approximately 1 meter tall, lightweight, quiet, and features no dangerous edges and soft skin. Its appearance and interactions are designed to be "approachable," like a social companion. It is currently offered as a general-purpose robotics platform for developers, providing basic movement, perception, navigation, and expression capabilities, allowing real-world use cases to emerge within the community. Many humanoid robots are born in factories and warehouses, but Fauna Robotics aims to bring humanoid robots back into human living spaces. Fauna believes that if humanoid robots are to become as ubiquitous as computers and mobile phones in the future, the starting point must be a safe, friendly form suitable for homes, schools, and offices. Sprout is their first step in this direction.

CyberRobo

19,516 просмотров • 9 месяцев назад

Training humanoid robots on teleoperation alone won’t scale, says Rhoda AI CEO Jagdeep Singh. Unlike self-driving cars - which basically have four actuators (left, right, speed up, and slow down) and operate in a single environment (the road) - humanoid robots are completely different: "You're dealing with the full dexterity of a human hand - 20 degrees of freedom per hand. Every object is different. Every type of task is different." The problem isn't just the quantity of data - the bigger issue is diversity in the data, and it's why many humanoid robot demos will struggle to adapt to the real world: "If all the data you have is data that you've intentionally collected, then you almost by definition haven't seen the corner cases. You haven't seen all those edge scenarios that cause failure." Jagdeep says teleoperation is useful for fine-tuning robot behavior, but for pretraining, it's completely inadequate.
2:46

Training humanoid robots on teleoperation alone won’t scale, says Rhoda AI CEO Jagdeep Singh. Unlike self-driving cars - which basically have four actuators (left, right, speed up, and slow down) and operate in a single environment (the road) - humanoid robots are completely different: "You're dealing with the full dexterity of a human hand - 20 degrees of freedom per hand. Every object is different. Every type of task is different." The problem isn't just the quantity of data - the bigger issue is diversity in the data, and it's why many humanoid robot demos will struggle to adapt to the real world: "If all the data you have is data that you've intentionally collected, then you almost by definition haven't seen the corner cases. You haven't seen all those edge scenarios that cause failure." Jagdeep says teleoperation is useful for fine-tuning robot behavior, but for pretraining, it's completely inadequate.

TBPN

19,218 просмотров • 7 месяцев назад

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:47

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

50,130 просмотров • 10 месяцев назад

Jensen: The humanoid robot is likely the only robot that will work – because technology needs scale, and most robots we've had so far are too low volume to drive the flywheel of technology improvements. The humanoid robot is likely to be the next multi-trillion-dollar industry.
1:26

Jensen: The humanoid robot is likely the only robot that will work – because technology needs scale, and most robots we've had so far are too low volume to drive the flywheel of technology improvements. The humanoid robot is likely to be the next multi-trillion-dollar industry.

The Humanoid Hub

84,720 просмотров • 1 год назад

Two wheeled humanoid robots are being deployed at JD's 7FRESH supermarket in Taiyuan… They are assisting with restocking, shelving, and providing product tasting and guidance. (VR TeleOP) From Beijing to Tianjin and Taiyuan…JD is introducing humanoid robot employees to its retail supermarkets in large numbers to enhance the shopping experience and alleviate peak customer traffic. But now,more importantly,TeleOP data These humanoid robots are from Beijing-based DEXTELEOP's TA2… who have also entered into a strategic partnership with JD. They are being deployed through JD's nationwide retail channels(20k). Placing data collection in real-world operational scenarios and achieving commercial success is a significant achievement. The physical world is the largest data collection center for humanoid robots.
2:50

Two wheeled humanoid robots are being deployed at JD's 7FRESH supermarket in Taiyuan… They are assisting with restocking, shelving, and providing product tasting and guidance. (VR TeleOP) From Beijing to Tianjin and Taiyuan…JD is introducing humanoid robot employees to its retail supermarkets in large numbers to enhance the shopping experience and alleviate peak customer traffic. But now,more importantly,TeleOP data These humanoid robots are from Beijing-based DEXTELEOP's TA2… who have also entered into a strategic partnership with JD. They are being deployed through JD's nationwide retail channels(20k). Placing data collection in real-world operational scenarios and achieving commercial success is a significant achievement. The physical world is the largest data collection center for humanoid robots.

CyberRobo

12,941 просмотров • 2 месяцев назад

Yann LeCun says today's humanoid robot demos look impressive, but the robots are very stupid in the real world "the missing piece is not hardware; it is AI that can't reason, plan, or adapt like humans" Robot companies are betting AI will make them smart enough to sell at scale within 3-5 years It's a big bet
1:19

Yann LeCun says today's humanoid robot demos look impressive, but the robots are very stupid in the real world "the missing piece is not hardware; it is AI that can't reason, plan, or adapt like humans" Robot companies are betting AI will make them smart enough to sell at scale within 3-5 years It's a big bet

Haider.

37,274 просмотров • 7 месяцев назад

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:56

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

21,042 просмотров • 8 месяцев назад

Jim Fan says NVIDIA trained humanoid robots to walk and move like humans -- zero-shot transfer from simulation to the real world. 10 years of learning, compressed into 2 hours of simulation. Turns out you don’t need a giant model to master motion. “1.5 million parameters, not billion, to capture the subconscious processing of the human body.” Big embodiment, small brain.
1:26

Jim Fan says NVIDIA trained humanoid robots to walk and move like humans -- zero-shot transfer from simulation to the real world. 10 years of learning, compressed into 2 hours of simulation. Turns out you don’t need a giant model to master motion. “1.5 million parameters, not billion, to capture the subconscious processing of the human body.” Big embodiment, small brain.

vitrupo

715,073 просмотров • 1 год назад

🚨🚨🚨Humanoid Robots Just Stepped Into the Ring** This is not really a fight. Not yet. The robots are slow, awkward, unstable, and clearly still learning how to move like athletes. But that is exactly what makes the clip interesting. A boxing ring is one of the hardest places to fake progress. Balance matters. Reaction time matters. Footwork matters. One bad movement, and the machine is on the floor. Today, it looks funny. But every fall is data. Every awkward punch is training. And every public demo like this brings humanoid robots one step closer to moving through the world with real physical confidence. The fight is not between two robots. It is between robotics and gravity.
0:25

🚨🚨🚨Humanoid Robots Just Stepped Into the Ring** This is not really a fight. Not yet. The robots are slow, awkward, unstable, and clearly still learning how to move like athletes. But that is exactly what makes the clip interesting. A boxing ring is one of the hardest places to fake progress. Balance matters. Reaction time matters. Footwork matters. One bad movement, and the machine is on the floor. Today, it looks funny. But every fall is data. Every awkward punch is training. And every public demo like this brings humanoid robots one step closer to moving through the world with real physical confidence. The fight is not between two robots. It is between robotics and gravity.

velora

15,097 просмотров • 2 месяцев назад

11 years of humanoid robotics in one video. 🤖 In 2015, humanoid robots were still struggling with basic tasks like walking, climbing stairs and recovering from falls during the DARPA Robotics Challenge. Fast-forward to 2026, and humanoids are running, jumping, boxing, playing football, handling industrial tasks and operating in real-world environments. The World Humanoid Robot Games featured 2,056 robots across 51 events, including practical tasks in factories, hotels and homes. The progress is obvious. But the real question is not how impressive they look in a demo. What will humanoid robots be capable of in 2036? 👀 🎥 Media: Defense Advanced Research Projects Agency (DARPA) / Unitree Robotics ⚠️ This content is shared for informational purposes only. CTO Robotics Media is a media platform and does not own or develop the technologies shown. Credit belongs to the original creators. #HumanoidRobots #Robotics #PhysicalAI #AI #DARPA #RobotEvolution #FutureOfRobotics #EmbodiedAI #Automation #RobotTechnology #CTORoboticsMedia
0:39

11 years of humanoid robotics in one video. 🤖 In 2015, humanoid robots were still struggling with basic tasks like walking, climbing stairs and recovering from falls during the DARPA Robotics Challenge. Fast-forward to 2026, and humanoids are running, jumping, boxing, playing football, handling industrial tasks and operating in real-world environments. The World Humanoid Robot Games featured 2,056 robots across 51 events, including practical tasks in factories, hotels and homes. The progress is obvious. But the real question is not how impressive they look in a demo. What will humanoid robots be capable of in 2036? 👀 🎥 Media: Defense Advanced Research Projects Agency (DARPA) / Unitree Robotics ⚠️ This content is shared for informational purposes only. CTO Robotics Media is a media platform and does not own or develop the technologies shown. Credit belongs to the original creators. #HumanoidRobots #Robotics #PhysicalAI #AI #DARPA #RobotEvolution #FutureOfRobotics #EmbodiedAI #Automation #RobotTechnology #CTORoboticsMedia

CTO ROBOTICS Media

337,033 просмотров • 1 месяц назад

The first factory in the world for the production of humanoid robots has opened in China, where "robots produce robots." Assembling one robot will take 10 minutes. It is planned to produce up to 10,000 humanoid robots annually.
0:31

The first factory in the world for the production of humanoid robots has opened in China, where "robots produce robots." Assembling one robot will take 10 minutes. It is planned to produce up to 10,000 humanoid robots annually.

𝕊𝕡𝕣𝕚𝕟𝕥𝕖𝕣 𝕻𝕣𝕖𝕤𝕤

12,407 просмотров • 1 месяц назад