Humanoid motion tracking performance is greatly determined by retargeting quality! Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL: - 5 rewards, - 4 DR terms, - Proprio. ONLY, - NO history/curriculum. Ready for agile, human-like 🤖? (Best with 🎧) 🔗... 🎥 1/9show more

Existing retargeting often produces artifacts like foot-skating and penetration ❌. To compensate, RL policies rely on complex ad-hoc reward terms, forcing a trade-off between accurate motion tracking and correcting errors like slipping or bad contacts. OmniRetarget fixes this at the source! ✅ Using an "interaction mesh," it generates clean, physically feasible trajectories by explicitly preserving the spatial and contact relationships between the agent, terrain, and objects. ✨ 2/9

The result of this high-quality data? We can train diverse skills like box carrying 📦, slope crawling 🐾, and platform climbing 🧗 with a radically simplified RL process! All policies use just 5 reward terms, achieving successful zero-shot sim-to-real transfer! 🎯➡️🦾 3/9

What about scalability? OmniRetarget transforms a SINGLE human demo into diverse motion clips. We can systematically vary terrain height, object size, and initial poses. Best of all, these augmented skills transfer directly from sim to our real-world hardware! 🤖➡️🦾 4/9

And it's not just for a specific robot! Our framework is highly general and adapts to different robot embodiments, including the @UnitreeRobotics H1 and the @boosterobotics T1. We can retarget complex object-carrying and platform-climbing skills across these different robots with minimal changes. 5/9

But how much better is our data? 🤔 Compared to widely-used baselines, our motions show far fewer physical artifacts—virtually zero foot-skating and penetration—while better preserving contact. This allows us to use an open-sourced RL framework (BeyondMimic) without hyperparameters tuning, while baselines fail to achieve high success rates in this setting. 6/9

Our grand finale: A complex, long-horizon dynamic sequence, all driven by a proprioceptive-only policy (no vision/LIDAR)! In this task, the robot carries a chair to a platform, uses it as a step to climb up, then leaps off and performs a parkour-style roll to absorb the landing. This pushes the boundaries of agile, human-like loco-manipulation! 7/9

Standing on the shoulders of giants! Our work builds on amazing research in the community💡. We use the "interaction mesh" 🕸️ [1], [2] to preserve spatial relationships and leverage the minimal RL formulation from works like BeyondMimic [3]. Our long-horizon sequence is a nod to the incredible Boston Dynamics Atlas demos 🤖 [4]! [1] E. S. L. Ho, T. Komura, and C.-L. Tai, “Spatial relationship preserving character motion adaptation,” ACM Transactions on Graphics, 2010. [2] S. Nakaoka and T. Komura, “Interaction mesh based motion adaptation for biped humanoid robots,” in Humanoids, 2012. [3] Q. Liao, T. E. Truong, X. Huang, G. Tevet, K. Sreenath, and C. K. Liu, “Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion,” arXiv e-prints, pp. arXiv–2508, 2025. [4] Boston Dynamics, “Atlas Gets a Grip,” YouTube, available: https:// 8/9

We are open-sourcing over 4 hours of high-quality, retargeted trajectories! Website: ArXiv: Datasets: Huge shout out to the amazing team: @lujieyang98, @x_h_ucb, @akanazawa, @pabbeel, @carlo_sferrazza, @ckarenliu, @rocky_duan, and @GuanyaShi from Amazon FAR, MIT, UC Berkeley, Stanford, and CMU! This work was done during an internship at Amazon FAR (Frontier AI & Robotics). 9/9

climbed like a real human. Soon, we will see robot parkour competitions.

Beautiful results!!! And cliffhanger 😂

Thanks Brent! Yeah there are more exciting parkour-style motions on the way 🚗🚗🚗😝

Very cool work

I don’t know you personally yet, but you superstar better than 1000 Kardashians and Ronaldo 🚀🤖

Incredible work! Really solid result

@Scobleizer This looks like a huge step forward for more natural humanoid motion 👏

Impressive work! Fixing retargeting artifacts at the source rather than with complex reward engineering is the right approach. The long-horizon parkour sequence is stunning!

This is truly impressive to see how generalizeable this is and also simplifies the skill transfer process to potentially hundreds of humanoid robot vendors. Won’t be surprised to see lots of robotics companies building upon this work in the future.

this is what the community needs:) and these videos are really impressive!

Time for #HumanoidGeneration

It is really good to see motion tracking improved like this. OmniRetarget proves how much the right starting point matters. That is also what AIOZ Movie Review Challenge does. It gives creators a fun and simple way to share movies and connect.

@grok What's the meaning of retargeting?

should fit

That’s some really nice work! Congratulations!

@TairanHe99 If you had to choose which is more efficient, learning from Third-Person Human or Learning from Motion Capture

Interesting work! Specially since it doesn’t need to undergo curriculum training. Could OmniRetarget be made adaptive to the downstream RL task or policy uncertainty, dynamically refining trajectories?

Wow interesting stuff and you say it's generalizable to other robots? Def interested 😁

wow!!! 🥹🫶 amazing job this is so cool!!

Pelvic and hip mobility is bloody amazing

Impressive work on solving those artifact issues! Streamlined RL with fewer reward terms is a game changer. What's next for scalability challenges?

Very impressive

@Tesla_Optimus

@Threadreaderapp unroll

Any real world applications such as construction or working a garbage sorting facility, parkour stuff is cool but I am seeking enlightenment elsewhere in reducing human toil

That's great.

Congrats! When will you open-source the motion retargeting code🥹