So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data... generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)show more

Step 1: Collect lots of environments with video scans - anyone can do it with their phone. I even had my parents scan in a bunch :) use 3D reconstruction methods like Gaussian splats to make diverse, visually & geometrically realistic sim environments for training policies (2/N)

Step 2: Train policies on these environments with demo-bootstrapped RL. A couple of demos are needed to guide exploration, but the heavy lifting is done with large scale RL in simulation . This takes success rates from 2-3% to >90% success from <10 human demos. (3/N)

Step 3: We realize that providing even 10 demos on each environment is still too expensive. By training vision based generalists from the RL data, we can get a degree of cross-environment generalization that allows the model to provide *itself* demos and only use human effort when the model generalization provides no success. The better the generalist policy gets, the less human effort is required. (4/N)

Step 4: Transfer over to the real world, either zero-shot or with some co-training. Shows scaling laws as more experience is encountered, and robust performance across distractors, object positions, visual conditions and disturbances for multiple manipulation tasks. (5/N)

Step 5: One neat feature is that in a test environment, human demos aren’t even required. Scan in just an environment video to build a test-time simulation and let the generalist model provide itself demos and improve with RL in sim. Results in over 50% improvement with 0 human effort (6/N)

Why do I care about this - I think we’re going to have to consider off-domain data for robotics, and realistic simulation constructed cheaply from video provides a scalable way to source and utilize this data. Building methods that scale sub linearly with human effort make this practical, and scaling across environments makes this generalizable. (7/N)

And finally, I’m a sucker for a fun website. Check out our interactive demo where you can see and interact with some of the environments and learned behaviors. We’ve also open sourced USDZ assets of the sourced environments. I’m excited about the doors this opens for generalizable robot pre-training and data scaling! For more details: Paper: Website: USD assets: Fun project w/ @marceltornev , @prodarhan , @carrieyuanjiayi, Vidyaaranya Macha, @larsankile , @anthonysimeono_ , @pulkitology. Looking forward to lots more real2sim work! (8/N)