Skip navigation
Robotics has a data scarcity problem - you simply can't scrape robot control data from webpages.
0:00 / 2:33

Robotics has a data scarcity problem - you simply can't scrape robot control data from webpages.

Jim Fan

@DrJimFan353K followers

Visit on X
1,857 likesDownloadOriginal post

163,670 views

Robotics has a data scarcity problem - you simply can't scrape robot control data from webpages. Introducing GR00T-Mimic and GR00T-Gen: using both Graphics 1.0 & Graphics 2.0 to multiply your robot datasets by 1,000,000x. We trade compute for synthetic data, so we are not capped by the fundamental physical limit of 24 hrs/robot/day. Robotics is right in the thick of Moravec's paradox: things that are easy for humans turn out to be incredibly hard for machines. We are crushing the Moravec's paradox, one token at a time. > Graphics 1.0: Isaac simulators with manually written, GPU-accelerated physics and rendering equations. > Graphics 2.0: big neural nets (Cosmos) that repaint the pixels from sim textures to real, given an open-ended prompt. Robot data multiplier workflow: 1. GR00T-Teleop: use XR device like Apple Vision Pro to map human finger poses to humanoid hands. 2. GR00T-Mimic: given a human-collected task demonstration, we augment the actions in Isaac and filter out ones that fail the task. 3. GR00T-Gen: apply Graphics 1.0 and then Graphics 2.0 to produce tons of visual variations. Join us to build Physical AI together! Job application links in thread:

11 Comments

Comments from the original post on X (Twitter)

  • Jim Fan's profile picture

    @DrJimFan

    We are hiring full-time researchers, engineers, and summer research interns!! If you want to get in touch and show us your past works, email [email protected] (me) and [email protected] (@yukez). Prefix your email subject with the literal "[NV GEAR APPLICATION]". Please forward these info to any friends you know who might be interested! - Sr. RE, Robot Systems - Sr. RE, Reinforcement Learning - Sr. RE, Foundation Model Training Infras - Sr. RE, Simulation - Sr. RE, ML Data Pipelines - Research Scientist - Research Intern

    129 likes

  • The Rundown AI's profile picture

    @TheRundownAI

    AI won't replace you, but a person using AI will. Join 500,000+ readers and learn how to use AI in just 5 minutes a day (for free).

    59,523 likes

  • Arnav Jaitly's profile picture

    @arnitly

    It’s cute how you found a way to name them Groot

    5 likes

  • Abhinav Girdhar's profile picture

    @AbhinavGirdhar

    Revolutionizing robotics data with GR00T-Mimic & GR00T-Gen—millions of synthetic points, smarter machines, and Moravec's paradox crushed!

    3 likes

  • The AI Veteran's profile picture

    @TheAIVeteran

    World sim and synthetic robot data generation in one day, and they already work together. This will help close the gap with Chinese robotics. Well done!

    2 likes

  • The Canaanite's profile picture

    @mysticaltech

    Incredible! Super smart synthetic data generation pipeline.

    2 likes

  • Elizabeth Greene's profile picture

    @GreeneElizabeth

    I don't understand why so many robots walk like they're doing the "gotta-poop-butt-clenched-shuffle." Is that a super-stable gait or somesuch?

    2 likes

  • Lukas Ziegler's profile picture

    @lukas_m_ziegler

    huuuuuge

    1 like

  • Prashant Rai's profile picture

    @imprashantrai1

    Who said robot learning is hitting the wall? It’s not just climbing over but smashing through!🙀

    1 like

  • jovin's profile picture

    @jovinxthomas

    This is an awesome explanation on why GR00T is so great! You all are doing amazing work at Nvidia! Definitely changing the world one token at a time 💪

    1 like

  • J⏩'s profile picture

    @JcardinaJ

    Please bring the background music down about 80% on future videos (or more ideally remove it entirely), it's competing with the message on my crappy (and typical) pc speaker.

    1 like