Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Not a preplanned motion sequence. A robot deciding mid-jump what to do next. [📍 paper + demo] Researchers just showed a humanoid doing real parkour using only onboard perception. No motion script, no fixed obstacle layout. The system is called Perceptive Humanoid Parkour (PHP). Instead of memorizing a path,... the robot reads depth from its cameras and continuously chooses actions. Step, vault, climb, or roll depending on what geometry appears in front of it. To make that possible, they combine three ideas: First, they stitch together human motion clips into long movement references so the robot learns fluid transitions instead of isolated tricks. Second, they train tracking policies with reinforcement learning so contacts land at the right time and the robot keeps balance during dynamic moves. Finally, everything is distilled into one perception policy that runs directly from depth input to action selection. The result on a Unitree G1: about 3 m/s vaults wall climbs up to 1.25 m nearly one minute continuous obstacle traversal adapting when obstacles move What matters is not the tricks. It is the shift in capability. Earlier humanoids executed motions. This one navigates situations. Once robots react to geometry instead of replaying trajectories, environments stop needing to be predictable. Warehouses, homes, and outdoors suddenly become the same problem. Thanks for sharing, Zhen Wu! Paper + demo: ——— Weekly robotics and AI insights. Subscribe free:show more

Ilir Aliu

52,211 subscribers

22,080 Aufrufe • vor 5 Monaten •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

Rohan Paul

37,121 Aufrufe • vor 5 Monaten

🚨 Apple joins the robotics race! Researchers from Carnegie Mellon University and Apple have developed a robot collision avoidance system. They called it ARMOR and they implemented it on the Fourier GR-1 robot. What is ARMOR? 👘 ARMOR is a system that helps humanoid robots "see" their surroundings better by using small, smart depth sensors placed on their arms. It solves the problem of blind spots and occlusions in traditional robot perception, making robots much safer and more efficient. It uses Time-of-Flight (ToF) sensors to detect obstacles, providing a near-complete view of the robot’s environment. 🔭 A smart software powered by transformer-based AI learns from human movements to help robots plan their actions dynamically. ARMOR reduces collisions by an impressive 63.7%, making robots significantly better at navigating crowded or complex spaces. The system is fast as well! 💨 ARMOR-Policy cuts computational time by 26× compared to traditional planning methods, ensuring robots can act quickly. P.S. Thoughts on this? The research paper is in the comments! ~~~ ♻️ RT to help 1 robot find a new workplace.

🚨 Apple joins the robotics race! Researchers from Carnegie Mellon University and Apple have developed a robot collision avoidance system. They called it ARMOR and they implemented it on the Fourier GR-1 robot. What is ARMOR? 👘 ARMOR is a system that helps humanoid robots "see" their surroundings better by using small, smart depth sensors placed on their arms. It solves the problem of blind spots and occlusions in traditional robot perception, making robots much safer and more efficient. It uses Time-of-Flight (ToF) sensors to detect obstacles, providing a near-complete view of the robot’s environment. 🔭 A smart software powered by transformer-based AI learns from human movements to help robots plan their actions dynamically. ARMOR reduces collisions by an impressive 63.7%, making robots significantly better at navigating crowded or complex spaces. The system is fast as well! 💨 ARMOR-Policy cuts computational time by 26× compared to traditional planning methods, ensuring robots can act quickly. P.S. Thoughts on this? The research paper is in the comments! ~~~ ♻️ RT to help 1 robot find a new workplace.

Lukas Ziegler

169,472 Aufrufe • vor 1 Jahr

The full LadderMan podcast episode is finally live! 🪜 LadderMan is one of the most interesting papers recently: instead of just walking on flat ground or doing flashy demos, it tackles a very real-world capability — letting humanoid robots climb ladders with perception, planning, and whole-body control. To me, this is exactly the kind of work that pushes robots closer to actually operating in human environments. Huge thanks again to Siheng Zhao , Chris Paxton, The Humanoid Hub, and Zifan Xu for joining the discussion. The full episode is a bit long, so I’ll be cutting some of the best moments into shorter clips and sharing them soon.

The full LadderMan podcast episode is finally live! 🪜 LadderMan is one of the most interesting papers recently: instead of just walking on flat ground or doing flashy demos, it tackles a very real-world capability — letting humanoid robots climb ladders with perception, planning, and whole-body control. To me, this is exactly the kind of work that pushes robots closer to actually operating in human environments. Huge thanks again to Siheng Zhao , Chris Paxton, The Humanoid Hub, and Zifan Xu for joining the discussion. The full episode is a bit long, so I’ll be cutting some of the best moments into shorter clips and sharing them soon.

Eren Chen

10,350 Aufrufe • vor 1 Monat

Turning video into humanoid robot motion! 🤳🏼 Training humanoid robots needs huge amounts of motion data, but real-world capture doesn’t scale. Mocap is expensive, dangerous edge cases are rare, and you can’t ask humans to repeatedly fall or crash. Video2Robot tackles this by converting videos into physics-grounded humanoid simulations. Motion is generated to respect balance, inertia, ground contact, and joint limits, then directly retargeted to robot simulators. One prompt can generate a full humanoid motion sequence, including multi-agent interactions and failure cases like falls or collisions, scenarios that are hard or impossible to capture safely in the real world. The pipeline is model-agnostic and works with existing video generators, making it a practical way to scale data for robots. If robots are going to operate in the real world, they need to be trained on the failures too, not just the perfect demos. Here's the GitHub: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Turning video into humanoid robot motion! 🤳🏼 Training humanoid robots needs huge amounts of motion data, but real-world capture doesn’t scale. Mocap is expensive, dangerous edge cases are rare, and you can’t ask humans to repeatedly fall or crash. Video2Robot tackles this by converting videos into physics-grounded humanoid simulations. Motion is generated to respect balance, inertia, ground contact, and joint limits, then directly retargeted to robot simulators. One prompt can generate a full humanoid motion sequence, including multi-agent interactions and failure cases like falls or collisions, scenarios that are hard or impossible to capture safely in the real world. The pipeline is model-agnostic and works with existing video generators, making it a practical way to scale data for robots. If robots are going to operate in the real world, they need to be trained on the failures too, not just the perfect demos. Here's the GitHub: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

59,835 Aufrufe • vor 7 Monaten

Twitter boi, do you want to see a humanoid AI lab from the inside? Well I do ;) Join me to visit Flexion A world-leading lab building the AI brains for humanoids – in Zurich, one of the best hubs for Physical AI. Their goal? Build the Android of robotics. One operating system that works on any humanoid hardware. It is crazy what their robots can do. We pushed a robot on the stairs. It didn't care. They left one in a rainy forest it was never trained on. Chilled and hiked there. And because they let robots train on their own, from zero through reinforcement learning, the robots do odd inhuman things. Eg they stand up from the ground in ways no human ever would. If their humanoid brain works, this will be industry defining. This is why Flexion is one of Europe's Most Ambitious Startups. 🇪🇺🔥 0:00 Intro 0:38 Inside a Lab with 14 Humanoids 2:17 What’s Actually Inside a Humanoid? 4:43 The Robot Navigates on Its Own 6:31 Why Grasping Is So Hard 7:01 Stairs, Pushes, and Balance Tests 7:55 Training 4,000 Robots in Parallel 10:50 Teleoperation vs Reinforcement Learning 14:35 The Case for Humanoids 15:55 Why Europe Could Win Robotics Please like & RT 🙏

Twitter boi, do you want to see a humanoid AI lab from the inside? Well I do ;) Join me to visit Flexion A world-leading lab building the AI brains for humanoids – in Zurich, one of the best hubs for Physical AI. Their goal? Build the Android of robotics. One operating system that works on any humanoid hardware. It is crazy what their robots can do. We pushed a robot on the stairs. It didn't care. They left one in a rainy forest it was never trained on. Chilled and hiked there. And because they let robots train on their own, from zero through reinforcement learning, the robots do odd inhuman things. Eg they stand up from the ground in ways no human ever would. If their humanoid brain works, this will be industry defining. This is why Flexion is one of Europe's Most Ambitious Startups. 🇪🇺🔥 0:00 Intro 0:38 Inside a Lab with 14 Humanoids 2:17 What’s Actually Inside a Humanoid? 4:43 The Robot Navigates on Its Own 6:31 Why Grasping Is So Hard 7:01 Stairs, Pushes, and Balance Tests 7:55 Training 4,000 Robots in Parallel 10:50 Teleoperation vs Reinforcement Learning 14:35 The Case for Humanoids 15:55 Why Europe Could Win Robotics Please like & RT 🙏

Andreas Klinger 🦾

14,761 Aufrufe • vor 4 Monaten

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Ilir Aliu - eu/acc

72,522 Aufrufe • vor 1 Jahr

These guys trained a robot with water bottles, and the robot learned to pick up other objects it had never practiced on. They did this using reinforcement learning: 1. The robot had to pick a bottle out of a messy tote 2. Then it had to hand the bottle to a person 3. Only water bottles. Nothing else. After they finished training, they tested the robot on different objects: • On bottles, the success went from 80% to 98% • On snack cans, the robot's throughput rose 40% • On soft bags, the robot's performance improved by 13% even though they're deformable and much harder to handle than rigid objects. Snack cans and soft bags have different shapes and weights, and the robot was never trained on them. This was just one of three use cases they tested. Every hour we spend teaching a robot costs a ton of money. If they only get better at handling the one object they practice on, reinforcement learning will never be worth doing. But if the gains carry over, we could focus on teaching the hardest possible tasks and let the robot apply that knowledge elsewhere. Here is the full write-up by Humanoid on the infrastructure they used and what they learned: #ad

These guys trained a robot with water bottles, and the robot learned to pick up other objects it had never practiced on. They did this using reinforcement learning: 1. The robot had to pick a bottle out of a messy tote 2. Then it had to hand the bottle to a person 3. Only water bottles. Nothing else. After they finished training, they tested the robot on different objects: • On bottles, the success went from 80% to 98% • On snack cans, the robot's throughput rose 40% • On soft bags, the robot's performance improved by 13% even though they're deformable and much harder to handle than rigid objects. Snack cans and soft bags have different shapes and weights, and the robot was never trained on them. This was just one of three use cases they tested. Every hour we spend teaching a robot costs a ton of money. If they only get better at handling the one object they practice on, reinforcement learning will never be worth doing. But if the gains carry over, we could focus on teaching the hardest possible tasks and let the robot apply that knowledge elsewhere. Here is the full write-up by Humanoid on the infrastructure they used and what they learned: #ad

Santiago

13,952 Aufrufe • vor 27 Tagen

New framework: Kick down your robot, it will get back up every time 🥋 Chinese startup RoboParty is a Beijing startup founded April 2025 by Huang Yi, originally shipping ROBOTO ORIGIN, the world's first full-stack open-source bipedal humanoid. They released UFO: Unsupervised Reinforcement Learning Framework for Humanoid Control. DEFINITIONS -> what differs is where the learning signal comes from: - SUPERVISED: humans supply the right answers (labels), the model imitates them. - UNSUPERVISED: no answer key, the model finds structure in raw data on its own. - REINFORCEMENT LEARNING: no answer key either, the model tries things and a reward scores each attempt. → UNSUPERVISED RL: trial and error where the agent invents its own rewards, instead of engineers hand-writing one per task. REPRESENTATION LEARNING: compress raw states into a useful internal map. TEMPORAL DISTANCE: distance on that map is "how many steps from A to B." CONTRASTIVE: trained by pulling together what's close in time, pushing apart what isn't. -> CONTRASTIVE TEMPORAL-DISTANCE REPRESENTATION LEARNING: the model builds an internal map of body states where distance means how many steps it takes to get from one to another. It is trained by contrast: states that occur close together in a movement get pulled together in the map, randomly paired states get pushed apart. UFO is an open-source training framework that teaches humanoid robots skills, like getting up, walking, goal-reaching, teleoperation, without reference motions -> no motion-capture or human-video demonstrations to imitate. Its core is TeCH, a contrastive temporal-distance representation-learning algorithm: the robot explores, builds pseudo-goals by temporal rolling, and learns goal-conditioned policies from a single unified progress reward. One framework trains five different robots (Unitree G1/H1, RoboParty RP0/RP1, AgiBot X2) with automatic config conversion in ~2–3 hours per robot! The real novelty here "no demonstrations at all". No data-collection arms race,the dominant humanoid-locomotion recipe is tracking: imitate mocap/retargeted-human reference trajectories. The robot self-generates goals from its own exploration and learns from a progress reward, needing zero reference motion data. Everybody else is fighting over data acquisition, while this team just teleports out of the race entirely (inb4 "competition is for losers 💀 ). This strategy reminds me of the DeepSeek playbook applied to robots: open-source the whole stack to become the global default and commoditize everyone else. RoboParty is giving away hardware and now control software (UFO) to be the Android of humanoids. Yet another reason for the US to ban Chinese open models perhaps 🥶 ? What I also really like about this approach is the cross-embodiment infrastructure, one framework trains Unitree G1/H1, RoboParty RP0/RP1, and AgiBot X2 with automatic configuration conversion. Just like Physical Intelligence, RoboParty seems to place itself as a neutral hardware agnostic middle man. Also woth mentioning: their ability ot perform stable skill injection, e.g. adding a cartwheel without forgetting how to walk. A common failure of RL humanoid policies is that teaching a new agile skill destabilizes the existing ones (catastrophic forgetting). UFO claims you can inject rare motions (cartwheel) without collapsing learned behavior. If it holds, that's a significant incremental/continual skill-learning! But again, I have to underline it: no arXiv, no external validation, no success-rate numbers. -> robotics badely needs an independent unbiased evaluator imho. Still, look at that cool demo: robot is getting kicked and pushed around (serious disturbance) during teleoperation (controlled the person at the back wearing the VR headset), and still managed to always get back up. This is some serious demonstration of stability and robustness!

New framework: Kick down your robot, it will get back up every time 🥋 Chinese startup RoboParty is a Beijing startup founded April 2025 by Huang Yi, originally shipping ROBOTO ORIGIN, the world's first full-stack open-source bipedal humanoid. They released UFO: Unsupervised Reinforcement Learning Framework for Humanoid Control. DEFINITIONS -> what differs is where the learning signal comes from: - SUPERVISED: humans supply the right answers (labels), the model imitates them. - UNSUPERVISED: no answer key, the model finds structure in raw data on its own. - REINFORCEMENT LEARNING: no answer key either, the model tries things and a reward scores each attempt. → UNSUPERVISED RL: trial and error where the agent invents its own rewards, instead of engineers hand-writing one per task. REPRESENTATION LEARNING: compress raw states into a useful internal map. TEMPORAL DISTANCE: distance on that map is "how many steps from A to B." CONTRASTIVE: trained by pulling together what's close in time, pushing apart what isn't. -> CONTRASTIVE TEMPORAL-DISTANCE REPRESENTATION LEARNING: the model builds an internal map of body states where distance means how many steps it takes to get from one to another. It is trained by contrast: states that occur close together in a movement get pulled together in the map, randomly paired states get pushed apart. UFO is an open-source training framework that teaches humanoid robots skills, like getting up, walking, goal-reaching, teleoperation, without reference motions -> no motion-capture or human-video demonstrations to imitate. Its core is TeCH, a contrastive temporal-distance representation-learning algorithm: the robot explores, builds pseudo-goals by temporal rolling, and learns goal-conditioned policies from a single unified progress reward. One framework trains five different robots (Unitree G1/H1, RoboParty RP0/RP1, AgiBot X2) with automatic config conversion in ~2–3 hours per robot! The real novelty here "no demonstrations at all". No data-collection arms race,the dominant humanoid-locomotion recipe is tracking: imitate mocap/retargeted-human reference trajectories. The robot self-generates goals from its own exploration and learns from a progress reward, needing zero reference motion data. Everybody else is fighting over data acquisition, while this team just teleports out of the race entirely (inb4 "competition is for losers 💀 ). This strategy reminds me of the DeepSeek playbook applied to robots: open-source the whole stack to become the global default and commoditize everyone else. RoboParty is giving away hardware and now control software (UFO) to be the Android of humanoids. Yet another reason for the US to ban Chinese open models perhaps 🥶 ? What I also really like about this approach is the cross-embodiment infrastructure, one framework trains Unitree G1/H1, RoboParty RP0/RP1, and AgiBot X2 with automatic configuration conversion. Just like Physical Intelligence, RoboParty seems to place itself as a neutral hardware agnostic middle man. Also woth mentioning: their ability ot perform stable skill injection, e.g. adding a cartwheel without forgetting how to walk. A common failure of RL humanoid policies is that teaching a new agile skill destabilizes the existing ones (catastrophic forgetting). UFO claims you can inject rare motions (cartwheel) without collapsing learned behavior. If it holds, that's a significant incremental/continual skill-learning! But again, I have to underline it: no arXiv, no external validation, no success-rate numbers. -> robotics badely needs an independent unbiased evaluator imho. Still, look at that cool demo: robot is getting kicked and pushed around (serious disturbance) during teleoperation (controlled the person at the back wearing the VR headset), and still managed to always get back up. This is some serious demonstration of stability and robustness!

Léo

35,705 Aufrufe • vor 12 Tagen

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

Ilir Aliu - eu/acc

10,509 Aufrufe • vor 1 Jahr

We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop. Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate. Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution. Our recipe is called "EgoScale": - Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks. - Most surprising result: a *single* teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency. - Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone. The scalable path to robot dexterity was never more robots. It was always us. Deep dives in thread:

We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop. Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate. Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution. Our recipe is called "EgoScale": - Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks. - Most surprising result: a single teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency. - Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone. The scalable path to robot dexterity was never more robots. It was always us. Deep dives in thread:

Jim Fan

294,395 Aufrufe • vor 5 Monaten

This is one of the more unusual flying robots I have seen. Researchers at the University of Tokyo built DRAGON, a transformable aerial robot that can change its shape while flying. Instead of a rigid frame, it is made of four linked segments. Each segment has two ducted rotors mounted on gimbals, so every link can redirect thrust independently. • The robot controls its full pose in SE(3) while reshaping mid-air • It can form different geometries to stabilize itself or interact with objects • The navigation stack computes the most efficient shape for the current task • It lifts over 3 kg, which is unusually high for this class of system This effectively turns a drone into a flying manipulator. Not a gripper bolted on top, but the airframe itself becomes the manipulator. The authors also mention an interesting idea for range extension: letting the robot walk on the ground when flying is not required. Paper ——- Weekly robotics and AI insights. Subscribe free:

This is one of the more unusual flying robots I have seen. Researchers at the University of Tokyo built DRAGON, a transformable aerial robot that can change its shape while flying. Instead of a rigid frame, it is made of four linked segments. Each segment has two ducted rotors mounted on gimbals, so every link can redirect thrust independently. • The robot controls its full pose in SE(3) while reshaping mid-air • It can form different geometries to stabilize itself or interact with objects • The navigation stack computes the most efficient shape for the current task • It lifts over 3 kg, which is unusually high for this class of system This effectively turns a drone into a flying manipulator. Not a gripper bolted on top, but the airframe itself becomes the manipulator. The authors also mention an interesting idea for range extension: letting the robot walk on the ground when flying is not required. Paper ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

39,213 Aufrufe • vor 3 Tagen

Why do most humanoid motion trackers still slip, skate, or clip through objects? Because the real bottleneck isn’t RL, it’s bad retargeting… OmniRetarget flips the script: instead of compensating with dozens of reward terms, it generates high-quality, interaction-preserving trajectories from human motions. The result is agile, human-like skills with minimal RL. ✅ 5 reward terms, 4 domain randomization terms, proprioception only, no history or curriculum ✅ Interaction mesh produces clean, contact-preserving motions with virtually no foot-skating ✅ Works across embodiments (Unitree H1, Booster T1) and scales from one demo to many scenes From carrying boxes to slope crawling to parkour-style rolls, policies trained on OmniRetarget data transfer zero-shot from sim to real hardware. Thanks for sharing, Zhen Wu! 📍Paper: Website: Dataset:

Why do most humanoid motion trackers still slip, skate, or clip through objects? Because the real bottleneck isn’t RL, it’s bad retargeting… OmniRetarget flips the script: instead of compensating with dozens of reward terms, it generates high-quality, interaction-preserving trajectories from human motions. The result is agile, human-like skills with minimal RL. ✅ 5 reward terms, 4 domain randomization terms, proprioception only, no history or curriculum ✅ Interaction mesh produces clean, contact-preserving motions with virtually no foot-skating ✅ Works across embodiments (Unitree H1, Booster T1) and scales from one demo to many scenes From carrying boxes to slope crawling to parkour-style rolls, policies trained on OmniRetarget data transfer zero-shot from sim to real hardware. Thanks for sharing, Zhen Wu! 📍Paper: Website: Dataset:

Ilir Aliu - eu/acc

61,491 Aufrufe • vor 10 Monaten

I don’t know if we live in a Matrix, but I know for sure that robots will spend most of their lives in simulation. Let machines train machines. I’m excited to introduce DexMimicGen, a massive-scale synthetic data generator that enables a humanoid robot to learn complex skills from only a handful of human demonstrations. Yes, as few as 5! DexMimicGen addresses the biggest pain point in robotics: where do we get data? Unlike with LLMs, where vast amounts of texts are readily available, you cannot simply download motor control signals from the internet. So researchers teleoperate the robots to collect motion data via XR headsets. They have to repeat the same skill over and over and over again, because neural nets are data hungry. This is a very slow and uncomfortable process. At NVIDIA, we believe the majority of high-quality tokens for robot foundation models will come from simulation. What DexMimicGen does is to trade GPU compute time for human time. It takes one motion trajectory from human, and multiplies into 1000s of new trajectories. A robot brain trained on this augmented dataset will generalize far better in the real world. Think of DexMimicGen as a learning signal amplifier. It maps a small dataset to a large (de facto infinite) dataset, using physics simulation in the loop. In this way, we free humans from babysitting the bots all day. The future of robot data is generative. The future of the entire robot learning pipeline will also be generative. 🧵

I don’t know if we live in a Matrix, but I know for sure that robots will spend most of their lives in simulation. Let machines train machines. I’m excited to introduce DexMimicGen, a massive-scale synthetic data generator that enables a humanoid robot to learn complex skills from only a handful of human demonstrations. Yes, as few as 5! DexMimicGen addresses the biggest pain point in robotics: where do we get data? Unlike with LLMs, where vast amounts of texts are readily available, you cannot simply download motor control signals from the internet. So researchers teleoperate the robots to collect motion data via XR headsets. They have to repeat the same skill over and over and over again, because neural nets are data hungry. This is a very slow and uncomfortable process. At NVIDIA, we believe the majority of high-quality tokens for robot foundation models will come from simulation. What DexMimicGen does is to trade GPU compute time for human time. It takes one motion trajectory from human, and multiplies into 1000s of new trajectories. A robot brain trained on this augmented dataset will generalize far better in the real world. Think of DexMimicGen as a learning signal amplifier. It maps a small dataset to a large (de facto infinite) dataset, using physics simulation in the loop. In this way, we free humans from babysitting the bots all day. The future of robot data is generative. The future of the entire robot learning pipeline will also be generative. 🧵

Jim Fan

165,246 Aufrufe • vor 1 Jahr

There’s not enough hype about Flexion Robotics. They are literally one of the sexiest European companies these days! 🇪🇺 I had a chance to meet the founder Nikita last year and learn his story. He turned down offers from top physical AI companies to build his own. The reason was his specific vision of how robot intelligence should be built to actually scale. So he decided to build one himself. Flexion is building an ultimate brain for humanoid robots. Yesterday they released Reflect v1.0. It’s an intelligence platform for long-horizon autonomous missions. The system has three layers. At the top, a custom vision-language model acts as mission control, observing the robot's camera feed and replanning continuously. Below that, a motion layer combines a VLA trained on real-world data with reinforcement learning-based skills. It translates perception into navigation, scene interaction, and manipulation commands. At the bottom, a whole-body controller runs in real-time, maintaining balance while the robot navigates and performs tasks. I highly recommend them visiting Zurich! 🇨🇭 Nikita Rudin cheapeu bas on the launch! 🎩 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news → http://

There’s not enough hype about Flexion Robotics. They are literally one of the sexiest European companies these days! 🇪🇺 I had a chance to meet the founder Nikita last year and learn his story. He turned down offers from top physical AI companies to build his own. The reason was his specific vision of how robot intelligence should be built to actually scale. So he decided to build one himself. Flexion is building an ultimate brain for humanoid robots. Yesterday they released Reflect v1.0. It’s an intelligence platform for long-horizon autonomous missions. The system has three layers. At the top, a custom vision-language model acts as mission control, observing the robot's camera feed and replanning continuously. Below that, a motion layer combines a VLA trained on real-world data with reinforcement learning-based skills. It translates perception into navigation, scene interaction, and manipulation commands. At the bottom, a whole-body controller runs in real-time, maintaining balance while the robot navigates and performs tasks. I highly recommend them visiting Zurich! 🇨🇭 Nikita Rudin cheapeu bas on the launch! 🎩 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news → http://

Lukas Ziegler

23,160 Aufrufe • vor 1 Monat

JUST IN: Unitree Robotics just unveiled the GD01, a manned "mecha" robot that transforms between two legs and four legs in real time. This is not a concept. This is a product. Starting price: $574,000. This robot weighs 500kg, is built from high-strength alloy, and has a torso-mounted cockpit. The demo video shows it walking upright, knocking down a brick wall with its hand, then reconfiguring into a four-legged crawl. Unitree calls it the world's first mass-produced transformable mecha. But the real story here is the bigger picture. Chinese companies accounted for nearly 90% of global humanoid robot sales in 2025. Unitree alone shipped 5,500+ humanoid robots last year. In the same period, Tesla, Figure AI and Agility Robotics each shipped around 150. And Unitree is now filing for an IPO on Shanghai's Star Market, targeting 4.2 billion yuan, 85% of which goes straight into R&D. The West is building incredible humanoid robots. China is building them faster, cheaper and at a scale nobody else is close to matching. Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

JUST IN: Unitree Robotics just unveiled the GD01, a manned "mecha" robot that transforms between two legs and four legs in real time. This is not a concept. This is a product. Starting price: $574,000. This robot weighs 500kg, is built from high-strength alloy, and has a torso-mounted cockpit. The demo video shows it walking upright, knocking down a brick wall with its hand, then reconfiguring into a four-legged crawl. Unitree calls it the world's first mass-produced transformable mecha. But the real story here is the bigger picture. Chinese companies accounted for nearly 90% of global humanoid robot sales in 2025. Unitree alone shipped 5,500+ humanoid robots last year. In the same period, Tesla, Figure AI and Agility Robotics each shipped around 150. And Unitree is now filing for an IPO on Shanghai's Star Market, targeting 4.2 billion yuan, 85% of which goes straight into R&D. The West is building incredible humanoid robots. China is building them faster, cheaper and at a scale nobody else is close to matching. Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

11,580 Aufrufe • vor 3 Monaten

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

Pascal Bornet

186,551 Aufrufe • vor 4 Monaten

"A parcel with snacks has been delivered for Flexion. Retrieve it using the stairs and come up using the elevator. Then unpack it and place the items into the empty drawer on the shelf in the snack area." One instruction. No human operator. Everything that follows is autonomous. Today we're introducing Reflect v1.0, our robotics intelligence platform for long-horizon work. From a single natural-language command, the robot understands the task, navigates a multi-floor building, calls elevators, handles doors, uses tools to unpack a box, and puts the items away. The biggest shift in v1.0 is that we use reinforcement learning across every layer, from low-level control to high-level reasoning. Long-horizon autonomy is unforgiving. The robot must recover on its own when things don't go to plan because in the real world, they never do. Combining reasoning, perception, physical execution and runtime robustness into a single mission-capable system is the foundation required to solve humanoid autonomy. Our team is just getting started. #HumanoidRobots #Flexion

"A parcel with snacks has been delivered for Flexion. Retrieve it using the stairs and come up using the elevator. Then unpack it and place the items into the empty drawer on the shelf in the snack area." One instruction. No human operator. Everything that follows is autonomous. Today we're introducing Reflect v1.0, our robotics intelligence platform for long-horizon work. From a single natural-language command, the robot understands the task, navigates a multi-floor building, calls elevators, handles doors, uses tools to unpack a box, and puts the items away. The biggest shift in v1.0 is that we use reinforcement learning across every layer, from low-level control to high-level reasoning. Long-horizon autonomy is unforgiving. The robot must recover on its own when things don't go to plan because in the real world, they never do. Combining reasoning, perception, physical execution and runtime robustness into a single mission-capable system is the foundation required to solve humanoid autonomy. Our team is just getting started. #HumanoidRobots #Flexion

Flexion

140,233 Aufrufe • vor 1 Monat

Pliny the Liberator asked "what is on the horizon?" Well, yesterday I visited Silicon Valley's InOrbit robot space to learn more about what's next in robotics. This place has a ton of specialized robots that do a variety of things for a variety of different businesses. sierra catalina will be happy, not a humanoid in it yet. But what they really are doing, beyond a community center for the robotics industry, is building an orchestration system for warehouses and factories and we get a good look at that too. Founder/CEO Florian Pestoni gives me a tour. So, Pliny the Liberator 🐉󠅫󠄼󠄿󠅆󠄵󠄐󠅀󠄼󠄹󠄾󠅉󠅭, here's your answer to "what is on the horizon:" More robots. Learn more about InOrbit here:

Pliny the Liberator asked "what is on the horizon?" Well, yesterday I visited Silicon Valley's InOrbit robot space to learn more about what's next in robotics. This place has a ton of specialized robots that do a variety of things for a variety of different businesses. sierra catalina will be happy, not a humanoid in it yet. But what they really are doing, beyond a community center for the robotics industry, is building an orchestration system for warehouses and factories and we get a good look at that too. Founder/CEO Florian Pestoni gives me a tour. So, Pliny the Liberator 🐉󠅫󠄼󠄿󠅆󠄵󠄐󠅀󠄼󠄹󠄾󠅉󠅭, here's your answer to "what is on the horizon:" More robots. Learn more about InOrbit here:

Robert Scoble

69,807 Aufrufe • vor 1 Jahr

this is the world's first ever humanoid robot that will summit Mt. Everest it's named Pemba, and two days ago it reached the top of Chimborazo, a 20,000-foot peak in Ecuador, completely on its own (no remote or operator). and the robot itself is nothing special. Pemba is a Unitree G1, the same ~$14k robot anyone can buy online right now. the guy behind it, Pablo Berlanga, is doing this to send robots into the places that kill people. think about how you'd check on a melting glacier or a deadly crevasse out in the middle of nowhere today. you either send a person who might not come back, or you skip it and learn nothing. so Berlanga wants a robot that walks in on its own, carries a few pounds of gear, and brings back footage from places no human can safely reach. i think eventually they'll even be used for robot rescue missions to reach people stranded in disasters and dangerous situations from here, the plan for Pemba is Mauna Kea in Hawaii, then Everest. Everest is the tricky one. the team wanted to send Pemba up this spring to start hauling trash off the mountain and tracking its glaciers (something Nepal genuinely needs help with) but the Nepalese government told them to wait. there's no law in Nepal for a climber that isn't human, so the rules have to get written before a robot can set foot on the mountain. kind of incredible that the machine is ready for Everest a full year before anyone's decided whether it's allowed to be up there

this is the world's first ever humanoid robot that will summit Mt. Everest it's named Pemba, and two days ago it reached the top of Chimborazo, a 20,000-foot peak in Ecuador, completely on its own (no remote or operator). and the robot itself is nothing special. Pemba is a Unitree G1, the same ~$14k robot anyone can buy online right now. the guy behind it, Pablo Berlanga, is doing this to send robots into the places that kill people. think about how you'd check on a melting glacier or a deadly crevasse out in the middle of nowhere today. you either send a person who might not come back, or you skip it and learn nothing. so Berlanga wants a robot that walks in on its own, carries a few pounds of gear, and brings back footage from places no human can safely reach. i think eventually they'll even be used for robot rescue missions to reach people stranded in disasters and dangerous situations from here, the plan for Pemba is Mauna Kea in Hawaii, then Everest. Everest is the tricky one. the team wanted to send Pemba up this spring to start hauling trash off the mountain and tracking its glaciers (something Nepal genuinely needs help with) but the Nepalese government told them to wait. there's no law in Nepal for a climber that isn't human, so the rules have to get written before a robot can set foot on the mountain. kind of incredible that the machine is ready for Everest a full year before anyone's decided whether it's allowed to be up there

Ole Lehmann

33,968 Aufrufe • vor 2 Monaten

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

50,063 Aufrufe • vor 7 Monaten