Загрузка видео...

Не удалось загрузить видео

На главную

Not a preplanned motion sequence. A robot deciding mid-jump what to do next. [📍 paper + demo] Researchers just showed a humanoid doing real parkour using only onboard perception. No motion script, no fixed obstacle layout. The system is called Perceptive Humanoid Parkour (PHP). Instead of memorizing a path,...

IlirAliu_'s profile picture

Ilir Aliu

52,211 subscribers

22,080 просмотров • 6 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.
0:05

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

Rohan Paul

37,121 просмотров • 6 месяцев назад

🚨 Apple joins the robotics race! Researchers from Carnegie Mellon University and Apple have developed a robot collision avoidance system. They called it ARMOR and they implemented it on the Fourier GR-1 robot. What is ARMOR? 👘 ARMOR is a system that helps humanoid robots "see" their surroundings better by using small, smart depth sensors placed on their arms. It solves the problem of blind spots and occlusions in traditional robot perception, making robots much safer and more efficient. It uses Time-of-Flight (ToF) sensors to detect obstacles, providing a near-complete view of the robot’s environment. 🔭 A smart software powered by transformer-based AI learns from human movements to help robots plan their actions dynamically. ARMOR reduces collisions by an impressive 63.7%, making robots significantly better at navigating crowded or complex spaces. The system is fast as well! 💨 ARMOR-Policy cuts computational time by 26× compared to traditional planning methods, ensuring robots can act quickly. P.S. Thoughts on this? The research paper is in the comments! ~~~ ♻️ RT to help 1 robot find a new workplace.
1:31

🚨 Apple joins the robotics race! Researchers from Carnegie Mellon University and Apple have developed a robot collision avoidance system. They called it ARMOR and they implemented it on the Fourier GR-1 robot. What is ARMOR? 👘 ARMOR is a system that helps humanoid robots "see" their surroundings better by using small, smart depth sensors placed on their arms. It solves the problem of blind spots and occlusions in traditional robot perception, making robots much safer and more efficient. It uses Time-of-Flight (ToF) sensors to detect obstacles, providing a near-complete view of the robot’s environment. 🔭 A smart software powered by transformer-based AI learns from human movements to help robots plan their actions dynamically. ARMOR reduces collisions by an impressive 63.7%, making robots significantly better at navigating crowded or complex spaces. The system is fast as well! 💨 ARMOR-Policy cuts computational time by 26× compared to traditional planning methods, ensuring robots can act quickly. P.S. Thoughts on this? The research paper is in the comments! ~~~ ♻️ RT to help 1 robot find a new workplace.

Lukas Ziegler

169,472 просмотров • 1 год назад

The full LadderMan podcast episode is finally live! 🪜 LadderMan is one of the most interesting papers recently: instead of just walking on flat ground or doing flashy demos, it tackles a very real-world capability — letting humanoid robots climb ladders with perception, planning, and whole-body control. To me, this is exactly the kind of work that pushes robots closer to actually operating in human environments. Huge thanks again to Siheng Zhao , Chris Paxton, The Humanoid Hub, and Zifan Xu for joining the discussion. The full episode is a bit long, so I’ll be cutting some of the best moments into shorter clips and sharing them soon.
54:58

The full LadderMan podcast episode is finally live! 🪜 LadderMan is one of the most interesting papers recently: instead of just walking on flat ground or doing flashy demos, it tackles a very real-world capability — letting humanoid robots climb ladders with perception, planning, and whole-body control. To me, this is exactly the kind of work that pushes robots closer to actually operating in human environments. Huge thanks again to Siheng Zhao , Chris Paxton, The Humanoid Hub, and Zifan Xu for joining the discussion. The full episode is a bit long, so I’ll be cutting some of the best moments into shorter clips and sharing them soon.

Eren Chen

10,612 просмотров • 2 месяцев назад

Turning video into humanoid robot motion! 🤳🏼 Training humanoid robots needs huge amounts of motion data, but real-world capture doesn’t scale. Mocap is expensive, dangerous edge cases are rare, and you can’t ask humans to repeatedly fall or crash. Video2Robot tackles this by converting videos into physics-grounded humanoid simulations. Motion is generated to respect balance, inertia, ground contact, and joint limits, then directly retargeted to robot simulators. One prompt can generate a full humanoid motion sequence, including multi-agent interactions and failure cases like falls or collisions, scenarios that are hard or impossible to capture safely in the real world. The pipeline is model-agnostic and works with existing video generators, making it a practical way to scale data for robots. If robots are going to operate in the real world, they need to be trained on the failures too, not just the perfect demos. Here's the GitHub: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:26

Turning video into humanoid robot motion! 🤳🏼 Training humanoid robots needs huge amounts of motion data, but real-world capture doesn’t scale. Mocap is expensive, dangerous edge cases are rare, and you can’t ask humans to repeatedly fall or crash. Video2Robot tackles this by converting videos into physics-grounded humanoid simulations. Motion is generated to respect balance, inertia, ground contact, and joint limits, then directly retargeted to robot simulators. One prompt can generate a full humanoid motion sequence, including multi-agent interactions and failure cases like falls or collisions, scenarios that are hard or impossible to capture safely in the real world. The pipeline is model-agnostic and works with existing video generators, making it a practical way to scale data for robots. If robots are going to operate in the real world, they need to be trained on the failures too, not just the perfect demos. Here's the GitHub: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

59,835 просмотров • 8 месяцев назад

NVIDIA MotionBricks brings humanoid robots closer to natural, fluid movement The system combines a modular latent generative model with Smart Primitives, packing more than 350,000 motion skills into a single backbone. It runs in real time at up to 15,000 FPS with just 2ms latency. Navigation, object interaction, sitting, jumping, obstacle crossing, and even different movement styles can be composed zero-shot, like reusable motion modules. It is already being used for whole-body control on the Unitree G1 and serves as the motion layer for NVIDIA GR00T Whole-Body Control. What makes this interesting is the move from training individual behaviors toward building a shared library of motion capabilities. If that approach scales, robot movement could become more modular, transferable, and easier to deploy across different tasks and environments.
1:43

NVIDIA MotionBricks brings humanoid robots closer to natural, fluid movement The system combines a modular latent generative model with Smart Primitives, packing more than 350,000 motion skills into a single backbone. It runs in real time at up to 15,000 FPS with just 2ms latency. Navigation, object interaction, sitting, jumping, obstacle crossing, and even different movement styles can be composed zero-shot, like reusable motion modules. It is already being used for whole-body control on the Unitree G1 and serves as the motion layer for NVIDIA GR00T Whole-Body Control. What makes this interesting is the move from training individual behaviors toward building a shared library of motion capabilities. If that approach scales, robot movement could become more modular, transferable, and easier to deploy across different tasks and environments.

CyberRobo

11,089 просмотров • 2 месяцев назад

⚠️ THE ROBOT DIDN’T ENTER A ROBOT TESTING LAB. IT ENTERED A CITY. A humanoid robot is moving through a real urban environment with pedestrians, traffic and constantly changing space around it. There’s no empty runway and no perfectly controlled path - the machine has to keep moving while the world around it does whatever it wants. That makes a few seconds of walking far more interesting than another polished stage demo. A city is one of the hardest places to test a humanoid because nothing stays predictable for long. People stop, cars move, distances change and obstacles appear without warning, while the robot still has to maintain balance and choose where to step next. The challenge isn't simply walking anymore - it's understanding how to move through a world that wasn't designed for robots. And that's the real milestone here. Humanoids don't need to look perfectly human to change the world; they need to become capable of functioning in spaces already built for us. The closer they get to doing that reliably, the less important the laboratory demos become. The future won't feel futuristic because robots look human. It will feel futuristic when robots can move through our world like they belong there.
0:23

⚠️ THE ROBOT DIDN’T ENTER A ROBOT TESTING LAB. IT ENTERED A CITY. A humanoid robot is moving through a real urban environment with pedestrians, traffic and constantly changing space around it. There’s no empty runway and no perfectly controlled path - the machine has to keep moving while the world around it does whatever it wants. That makes a few seconds of walking far more interesting than another polished stage demo. A city is one of the hardest places to test a humanoid because nothing stays predictable for long. People stop, cars move, distances change and obstacles appear without warning, while the robot still has to maintain balance and choose where to step next. The challenge isn't simply walking anymore - it's understanding how to move through a world that wasn't designed for robots. And that's the real milestone here. Humanoids don't need to look perfectly human to change the world; they need to become capable of functioning in spaces already built for us. The closer they get to doing that reliably, the less important the laboratory demos become. The future won't feel futuristic because robots look human. It will feel futuristic when robots can move through our world like they belong there.

Future Memo

10,138 просмотров • 15 дней назад

Twitter boi, do you want to see a humanoid AI lab from the inside? Well I do ;) Join me to visit Flexion A world-leading lab building the AI brains for humanoids – in Zurich, one of the best hubs for Physical AI. Their goal? Build the Android of robotics. One operating system that works on any humanoid hardware. It is crazy what their robots can do. We pushed a robot on the stairs. It didn't care. They left one in a rainy forest it was never trained on. Chilled and hiked there. And because they let robots train on their own, from zero through reinforcement learning, the robots do odd inhuman things. Eg they stand up from the ground in ways no human ever would. If their humanoid brain works, this will be industry defining. This is why Flexion is one of Europe's Most Ambitious Startups. 🇪🇺🔥 0:00 Intro 0:38 Inside a Lab with 14 Humanoids 2:17 What’s Actually Inside a Humanoid? 4:43 The Robot Navigates on Its Own 6:31 Why Grasping Is So Hard 7:01 Stairs, Pushes, and Balance Tests 7:55 Training 4,000 Robots in Parallel 10:50 Teleoperation vs Reinforcement Learning 14:35 The Case for Humanoids 15:55 Why Europe Could Win Robotics Please like & RT 🙏
18:51

Twitter boi, do you want to see a humanoid AI lab from the inside? Well I do ;) Join me to visit Flexion A world-leading lab building the AI brains for humanoids – in Zurich, one of the best hubs for Physical AI. Their goal? Build the Android of robotics. One operating system that works on any humanoid hardware. It is crazy what their robots can do. We pushed a robot on the stairs. It didn't care. They left one in a rainy forest it was never trained on. Chilled and hiked there. And because they let robots train on their own, from zero through reinforcement learning, the robots do odd inhuman things. Eg they stand up from the ground in ways no human ever would. If their humanoid brain works, this will be industry defining. This is why Flexion is one of Europe's Most Ambitious Startups. 🇪🇺🔥 0:00 Intro 0:38 Inside a Lab with 14 Humanoids 2:17 What’s Actually Inside a Humanoid? 4:43 The Robot Navigates on Its Own 6:31 Why Grasping Is So Hard 7:01 Stairs, Pushes, and Balance Tests 7:55 Training 4,000 Robots in Parallel 10:50 Teleoperation vs Reinforcement Learning 14:35 The Case for Humanoids 15:55 Why Europe Could Win Robotics Please like & RT 🙏

Andreas Klinger 🦾

14,761 просмотров • 5 месяцев назад

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:
0:42

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Ilir Aliu - eu/acc

72,522 просмотров • 1 год назад

These guys trained a robot with water bottles, and the robot learned to pick up other objects it had never practiced on. They did this using reinforcement learning: 1. The robot had to pick a bottle out of a messy tote 2. Then it had to hand the bottle to a person 3. Only water bottles. Nothing else. After they finished training, they tested the robot on different objects: • On bottles, the success went from 80% to 98% • On snack cans, the robot's throughput rose 40% • On soft bags, the robot's performance improved by 13% even though they're deformable and much harder to handle than rigid objects. Snack cans and soft bags have different shapes and weights, and the robot was never trained on them. This was just one of three use cases they tested. Every hour we spend teaching a robot costs a ton of money. If they only get better at handling the one object they practice on, reinforcement learning will never be worth doing. But if the gains carry over, we could focus on teaching the hardest possible tasks and let the robot apply that knowledge elsewhere. Here is the full write-up by Humanoid on the infrastructure they used and what they learned: #ad
1:40

These guys trained a robot with water bottles, and the robot learned to pick up other objects it had never practiced on. They did this using reinforcement learning: 1. The robot had to pick a bottle out of a messy tote 2. Then it had to hand the bottle to a person 3. Only water bottles. Nothing else. After they finished training, they tested the robot on different objects: • On bottles, the success went from 80% to 98% • On snack cans, the robot's throughput rose 40% • On soft bags, the robot's performance improved by 13% even though they're deformable and much harder to handle than rigid objects. Snack cans and soft bags have different shapes and weights, and the robot was never trained on them. This was just one of three use cases they tested. Every hour we spend teaching a robot costs a ton of money. If they only get better at handling the one object they practice on, reinforcement learning will never be worth doing. But if the gains carry over, we could focus on teaching the hardest possible tasks and let the robot apply that knowledge elsewhere. Here is the full write-up by Humanoid on the infrastructure they used and what they learned: #ad

Santiago

13,952 просмотров • 1 месяц назад

New framework: Kick down your robot, it will get back up every time 🥋 Chinese startup RoboParty is a Beijing startup founded April 2025 by Huang Yi, originally shipping ROBOTO ORIGIN, the world's first full-stack open-source bipedal humanoid. They released UFO: Unsupervised Reinforcement Learning Framework for Humanoid Control. DEFINITIONS -> what differs is where the learning signal comes from: - SUPERVISED: humans supply the right answers (labels), the model imitates them. - UNSUPERVISED: no answer key, the model finds structure in raw data on its own. - REINFORCEMENT LEARNING: no answer key either, the model tries things and a reward scores each attempt. → UNSUPERVISED RL: trial and error where the agent invents its own rewards, instead of engineers hand-writing one per task. REPRESENTATION LEARNING: compress raw states into a useful internal map. TEMPORAL DISTANCE: distance on that map is "how many steps from A to B." CONTRASTIVE: trained by pulling together what's close in time, pushing apart what isn't. -> CONTRASTIVE TEMPORAL-DISTANCE REPRESENTATION LEARNING: the model builds an internal map of body states where distance means how many steps it takes to get from one to another. It is trained by contrast: states that occur close together in a movement get pulled together in the map, randomly paired states get pushed apart. UFO is an open-source training framework that teaches humanoid robots skills, like getting up, walking, goal-reaching, teleoperation, without reference motions -> no motion-capture or human-video demonstrations to imitate. Its core is TeCH, a contrastive temporal-distance representation-learning algorithm: the robot explores, builds pseudo-goals by temporal rolling, and learns goal-conditioned policies from a single unified progress reward. One framework trains five different robots (Unitree G1/H1, RoboParty RP0/RP1, AgiBot X2) with automatic config conversion in ~2–3 hours per robot! The real novelty here "no demonstrations at all". No data-collection arms race,the dominant humanoid-locomotion recipe is tracking: imitate mocap/retargeted-human reference trajectories. The robot self-generates goals from its own exploration and learns from a progress reward, needing zero reference motion data. Everybody else is fighting over data acquisition, while this team just teleports out of the race entirely (inb4 "competition is for losers 💀 ). This strategy reminds me of the DeepSeek playbook applied to robots: open-source the whole stack to become the global default and commoditize everyone else. RoboParty is giving away hardware and now control software (UFO) to be the Android of humanoids. Yet another reason for the US to ban Chinese open models perhaps 🥶 ? What I also really like about this approach is the cross-embodiment infrastructure, one framework trains Unitree G1/H1, RoboParty RP0/RP1, and AgiBot X2 with automatic configuration conversion. Just like Physical Intelligence, RoboParty seems to place itself as a neutral hardware agnostic middle man. Also woth mentioning: their ability ot perform stable skill injection, e.g. adding a cartwheel without forgetting how to walk. A common failure of RL humanoid policies is that teaching a new agile skill destabilizes the existing ones (catastrophic forgetting). UFO claims you can inject rare motions (cartwheel) without collapsing learned behavior. If it holds, that's a significant incremental/continual skill-learning! But again, I have to underline it: no arXiv, no external validation, no success-rate numbers. -> robotics badely needs an independent unbiased evaluator imho. Still, look at that cool demo: robot is getting kicked and pushed around (serious disturbance) during teleoperation (controlled the person at the back wearing the VR headset), and still managed to always get back up. This is some serious demonstration of stability and robustness!
0:49

New framework: Kick down your robot, it will get back up every time 🥋 Chinese startup RoboParty is a Beijing startup founded April 2025 by Huang Yi, originally shipping ROBOTO ORIGIN, the world's first full-stack open-source bipedal humanoid. They released UFO: Unsupervised Reinforcement Learning Framework for Humanoid Control. DEFINITIONS -> what differs is where the learning signal comes from: - SUPERVISED: humans supply the right answers (labels), the model imitates them. - UNSUPERVISED: no answer key, the model finds structure in raw data on its own. - REINFORCEMENT LEARNING: no answer key either, the model tries things and a reward scores each attempt. → UNSUPERVISED RL: trial and error where the agent invents its own rewards, instead of engineers hand-writing one per task. REPRESENTATION LEARNING: compress raw states into a useful internal map. TEMPORAL DISTANCE: distance on that map is "how many steps from A to B." CONTRASTIVE: trained by pulling together what's close in time, pushing apart what isn't. -> CONTRASTIVE TEMPORAL-DISTANCE REPRESENTATION LEARNING: the model builds an internal map of body states where distance means how many steps it takes to get from one to another. It is trained by contrast: states that occur close together in a movement get pulled together in the map, randomly paired states get pushed apart. UFO is an open-source training framework that teaches humanoid robots skills, like getting up, walking, goal-reaching, teleoperation, without reference motions -> no motion-capture or human-video demonstrations to imitate. Its core is TeCH, a contrastive temporal-distance representation-learning algorithm: the robot explores, builds pseudo-goals by temporal rolling, and learns goal-conditioned policies from a single unified progress reward. One framework trains five different robots (Unitree G1/H1, RoboParty RP0/RP1, AgiBot X2) with automatic config conversion in ~2–3 hours per robot! The real novelty here "no demonstrations at all". No data-collection arms race,the dominant humanoid-locomotion recipe is tracking: imitate mocap/retargeted-human reference trajectories. The robot self-generates goals from its own exploration and learns from a progress reward, needing zero reference motion data. Everybody else is fighting over data acquisition, while this team just teleports out of the race entirely (inb4 "competition is for losers 💀 ). This strategy reminds me of the DeepSeek playbook applied to robots: open-source the whole stack to become the global default and commoditize everyone else. RoboParty is giving away hardware and now control software (UFO) to be the Android of humanoids. Yet another reason for the US to ban Chinese open models perhaps 🥶 ? What I also really like about this approach is the cross-embodiment infrastructure, one framework trains Unitree G1/H1, RoboParty RP0/RP1, and AgiBot X2 with automatic configuration conversion. Just like Physical Intelligence, RoboParty seems to place itself as a neutral hardware agnostic middle man. Also woth mentioning: their ability ot perform stable skill injection, e.g. adding a cartwheel without forgetting how to walk. A common failure of RL humanoid policies is that teaching a new agile skill destabilizes the existing ones (catastrophic forgetting). UFO claims you can inject rare motions (cartwheel) without collapsing learned behavior. If it holds, that's a significant incremental/continual skill-learning! But again, I have to underline it: no arXiv, no external validation, no success-rate numbers. -> robotics badely needs an independent unbiased evaluator imho. Still, look at that cool demo: robot is getting kicked and pushed around (serious disturbance) during teleoperation (controlled the person at the back wearing the VR headset), and still managed to always get back up. This is some serious demonstration of stability and robustness!

Léo

35,855 просмотров • 29 дней назад

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER
0:30

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

Ilir Aliu - eu/acc

10,509 просмотров • 1 год назад

We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop. Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate. Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution. Our recipe is called "EgoScale": - Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks. - Most surprising result: a *single* teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency. - Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone. The scalable path to robot dexterity was never more robots. It was always us. Deep dives in thread:
1:17

We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop. Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate. Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution. Our recipe is called "EgoScale": - Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks. - Most surprising result: a *single* teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency. - Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone. The scalable path to robot dexterity was never more robots. It was always us. Deep dives in thread:

Jim Fan

299,688 просмотров • 6 месяцев назад

Why do most humanoid motion trackers still slip, skate, or clip through objects? Because the real bottleneck isn’t RL, it’s bad retargeting… OmniRetarget flips the script: instead of compensating with dozens of reward terms, it generates high-quality, interaction-preserving trajectories from human motions. The result is agile, human-like skills with minimal RL. ✅ 5 reward terms, 4 domain randomization terms, proprioception only, no history or curriculum ✅ Interaction mesh produces clean, contact-preserving motions with virtually no foot-skating ✅ Works across embodiments (Unitree H1, Booster T1) and scales from one demo to many scenes From carrying boxes to slope crawling to parkour-style rolls, policies trained on OmniRetarget data transfer zero-shot from sim to real hardware. Thanks for sharing, Zhen Wu! 📍Paper: Website: Dataset:
0:25

Why do most humanoid motion trackers still slip, skate, or clip through objects? Because the real bottleneck isn’t RL, it’s bad retargeting… OmniRetarget flips the script: instead of compensating with dozens of reward terms, it generates high-quality, interaction-preserving trajectories from human motions. The result is agile, human-like skills with minimal RL. ✅ 5 reward terms, 4 domain randomization terms, proprioception only, no history or curriculum ✅ Interaction mesh produces clean, contact-preserving motions with virtually no foot-skating ✅ Works across embodiments (Unitree H1, Booster T1) and scales from one demo to many scenes From carrying boxes to slope crawling to parkour-style rolls, policies trained on OmniRetarget data transfer zero-shot from sim to real hardware. Thanks for sharing, Zhen Wu! 📍Paper: Website: Dataset:

Ilir Aliu - eu/acc

61,491 просмотров • 11 месяцев назад

I don’t know if we live in a Matrix, but I know for sure that robots will spend most of their lives in simulation. Let machines train machines. I’m excited to introduce DexMimicGen, a massive-scale synthetic data generator that enables a humanoid robot to learn complex skills from only a handful of human demonstrations. Yes, as few as 5! DexMimicGen addresses the biggest pain point in robotics: where do we get data? Unlike with LLMs, where vast amounts of texts are readily available, you cannot simply download motor control signals from the internet. So researchers teleoperate the robots to collect motion data via XR headsets. They have to repeat the same skill over and over and over again, because neural nets are data hungry. This is a very slow and uncomfortable process. At NVIDIA, we believe the majority of high-quality tokens for robot foundation models will come from simulation. What DexMimicGen does is to trade GPU compute time for human time. It takes one motion trajectory from human, and multiplies into 1000s of new trajectories. A robot brain trained on this augmented dataset will generalize far better in the real world. Think of DexMimicGen as a learning signal amplifier. It maps a small dataset to a large (de facto infinite) dataset, using physics simulation in the loop. In this way, we free humans from babysitting the bots all day. The future of robot data is generative. The future of the entire robot learning pipeline will also be generative. 🧵
0:58

I don’t know if we live in a Matrix, but I know for sure that robots will spend most of their lives in simulation. Let machines train machines. I’m excited to introduce DexMimicGen, a massive-scale synthetic data generator that enables a humanoid robot to learn complex skills from only a handful of human demonstrations. Yes, as few as 5! DexMimicGen addresses the biggest pain point in robotics: where do we get data? Unlike with LLMs, where vast amounts of texts are readily available, you cannot simply download motor control signals from the internet. So researchers teleoperate the robots to collect motion data via XR headsets. They have to repeat the same skill over and over and over again, because neural nets are data hungry. This is a very slow and uncomfortable process. At NVIDIA, we believe the majority of high-quality tokens for robot foundation models will come from simulation. What DexMimicGen does is to trade GPU compute time for human time. It takes one motion trajectory from human, and multiplies into 1000s of new trajectories. A robot brain trained on this augmented dataset will generalize far better in the real world. Think of DexMimicGen as a learning signal amplifier. It maps a small dataset to a large (de facto infinite) dataset, using physics simulation in the loop. In this way, we free humans from babysitting the bots all day. The future of robot data is generative. The future of the entire robot learning pipeline will also be generative. 🧵

Jim Fan

165,246 просмотров • 1 год назад

JUST IN: Unitree Robotics just unveiled the GD01, a manned "mecha" robot that transforms between two legs and four legs in real time. This is not a concept. This is a product. Starting price: $574,000. This robot weighs 500kg, is built from high-strength alloy, and has a torso-mounted cockpit. The demo video shows it walking upright, knocking down a brick wall with its hand, then reconfiguring into a four-legged crawl. Unitree calls it the world's first mass-produced transformable mecha. But the real story here is the bigger picture. Chinese companies accounted for nearly 90% of global humanoid robot sales in 2025. Unitree alone shipped 5,500+ humanoid robots last year. In the same period, Tesla, Figure AI and Agility Robotics each shipped around 150. And Unitree is now filing for an IPO on Shanghai's Star Market, targeting 4.2 billion yuan, 85% of which goes straight into R&D. The West is building incredible humanoid robots. China is building them faster, cheaper and at a scale nobody else is close to matching. Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
1:06

JUST IN: Unitree Robotics just unveiled the GD01, a manned "mecha" robot that transforms between two legs and four legs in real time. This is not a concept. This is a product. Starting price: $574,000. This robot weighs 500kg, is built from high-strength alloy, and has a torso-mounted cockpit. The demo video shows it walking upright, knocking down a brick wall with its hand, then reconfiguring into a four-legged crawl. Unitree calls it the world's first mass-produced transformable mecha. But the real story here is the bigger picture. Chinese companies accounted for nearly 90% of global humanoid robot sales in 2025. Unitree alone shipped 5,500+ humanoid robots last year. In the same period, Tesla, Figure AI and Agility Robotics each shipped around 150. And Unitree is now filing for an IPO on Shanghai's Star Market, targeting 4.2 billion yuan, 85% of which goes straight into R&D. The West is building incredible humanoid robots. China is building them faster, cheaper and at a scale nobody else is close to matching. Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

11,690 просмотров • 3 месяцев назад

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork
0:20

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

Pascal Bornet

186,700 просмотров • 4 месяцев назад

There’s not enough hype about Flexion Robotics. They are literally one of the sexiest European companies these days! 🇪🇺 I had a chance to meet the founder Nikita last year and learn his story. He turned down offers from top physical AI companies to build his own. The reason was his specific vision of how robot intelligence should be built to actually scale. So he decided to build one himself. Flexion is building an ultimate brain for humanoid robots. Yesterday they released Reflect v1.0. It’s an intelligence platform for long-horizon autonomous missions. The system has three layers. At the top, a custom vision-language model acts as mission control, observing the robot's camera feed and replanning continuously. Below that, a motion layer combines a VLA trained on real-world data with reinforcement learning-based skills. It translates perception into navigation, scene interaction, and manipulation commands. At the bottom, a whole-body controller runs in real-time, maintaining balance while the robot navigates and performs tasks. I highly recommend them visiting Zurich! 🇨🇭 Nikita Rudin cheapeu bas on the launch! 🎩 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news → http://
0:23

There’s not enough hype about Flexion Robotics. They are literally one of the sexiest European companies these days! 🇪🇺 I had a chance to meet the founder Nikita last year and learn his story. He turned down offers from top physical AI companies to build his own. The reason was his specific vision of how robot intelligence should be built to actually scale. So he decided to build one himself. Flexion is building an ultimate brain for humanoid robots. Yesterday they released Reflect v1.0. It’s an intelligence platform for long-horizon autonomous missions. The system has three layers. At the top, a custom vision-language model acts as mission control, observing the robot's camera feed and replanning continuously. Below that, a motion layer combines a VLA trained on real-world data with reinforcement learning-based skills. It translates perception into navigation, scene interaction, and manipulation commands. At the bottom, a whole-body controller runs in real-time, maintaining balance while the robot navigates and performs tasks. I highly recommend them visiting Zurich! 🇨🇭 Nikita Rudin cheapeu bas on the launch! 🎩 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news → http://

Lukas Ziegler

23,200 просмотров • 2 месяцев назад

15–30 minutes of real-world robot data. That's now enough to go from sim-to-real failure to working robot. Let’s see… You train a robot in simulation. You deploy it in the real world. It fails. The physics don't match. So you try to fine-tune it with real data, but… you never have enough real data, and the fine-tuning breaks everything the simulation taught it. SimDist fixes this with one key decision: don't transfer the policy. Transfer the world model. Keep the reward and value knowledge from simulation frozen. Only update the part that's actually wrong, how the robot predicts physics. Now the robot doesn't have to relearn the entire task in the real world. It already knows what success looks like. It just needs to correct its understanding of how the real world moves. The part that makes this work: they also trained on failures and recoveries; not just perfect demonstrations. Without that, the planner finds the gaps and exploits them. With it, the robot can tell a good future from a bad one. That's all it needs. Results on peg insertion, table leg assembly, locomotion on slippery and uneven surfaces. Tasks that require precision, force, and quick reaction. Thanks for sharing, Tyler Westenbroek ([Tyler Westenbroek]. Interactive visualization + paper: ——- Weekly robotics and AI insights. Subscribe free:
0:41

15–30 minutes of real-world robot data. That's now enough to go from sim-to-real failure to working robot. Let’s see… You train a robot in simulation. You deploy it in the real world. It fails. The physics don't match. So you try to fine-tune it with real data, but… you never have enough real data, and the fine-tuning breaks everything the simulation taught it. SimDist fixes this with one key decision: don't transfer the policy. Transfer the world model. Keep the reward and value knowledge from simulation frozen. Only update the part that's actually wrong, how the robot predicts physics. Now the robot doesn't have to relearn the entire task in the real world. It already knows what success looks like. It just needs to correct its understanding of how the real world moves. The part that makes this work: they also trained on failures and recoveries; not just perfect demonstrations. Without that, the planner finds the gaps and exploits them. With it, the robot can tell a good future from a bad one. That's all it needs. Results on peg insertion, table leg assembly, locomotion on slippery and uneven surfaces. Tasks that require precision, force, and quick reaction. Thanks for sharing, Tyler Westenbroek ([Tyler Westenbroek]. Interactive visualization + paper: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

14,222 просмотров • 3 месяцев назад

"A parcel with snacks has been delivered for Flexion. Retrieve it using the stairs and come up using the elevator. Then unpack it and place the items into the empty drawer on the shelf in the snack area." One instruction. No human operator. Everything that follows is autonomous. Today we're introducing Reflect v1.0, our robotics intelligence platform for long-horizon work. From a single natural-language command, the robot understands the task, navigates a multi-floor building, calls elevators, handles doors, uses tools to unpack a box, and puts the items away. The biggest shift in v1.0 is that we use reinforcement learning across every layer, from low-level control to high-level reasoning. Long-horizon autonomy is unforgiving. The robot must recover on its own when things don't go to plan because in the real world, they never do. Combining reasoning, perception, physical execution and runtime robustness into a single mission-capable system is the foundation required to solve humanoid autonomy. Our team is just getting started. #HumanoidRobots #Flexion
4:17

"A parcel with snacks has been delivered for Flexion. Retrieve it using the stairs and come up using the elevator. Then unpack it and place the items into the empty drawer on the shelf in the snack area." One instruction. No human operator. Everything that follows is autonomous. Today we're introducing Reflect v1.0, our robotics intelligence platform for long-horizon work. From a single natural-language command, the robot understands the task, navigates a multi-floor building, calls elevators, handles doors, uses tools to unpack a box, and puts the items away. The biggest shift in v1.0 is that we use reinforcement learning across every layer, from low-level control to high-level reasoning. Long-horizon autonomy is unforgiving. The robot must recover on its own when things don't go to plan because in the real world, they never do. Combining reasoning, perception, physical execution and runtime robustness into a single mission-capable system is the foundation required to solve humanoid autonomy. Our team is just getting started. #HumanoidRobots #Flexion

Flexion

141,227 просмотров • 2 месяцев назад