Introducing a new, fully open robotics dataset! - 76k episodes - 564 unique scenes - 100 contributors - 13 labs/institutions - 3 continents A short 🧵 on the backstory

When we started this effort, we were planning to ask only our lab members to collect data. But, we wanted the dataset to be useful, so we started asking other labs what kind of data would be useful to them.

Without any expectations, we also asked if those other labs were interested in contributing, just in case they were. To my surprise, every single lab was interested in helping out! I think this reflects true camaraderie in the robot learning research community. ❤️

We first agreed on a platform & collection protocol, and implemented the software stack (took quite some time). Then, we started data collection! Beyond collection, we had subteams for data processing (e.g. annotation, storage), policy training, visualization, and paper writing

See the paper for lots of visualizations & policy co-training results. Unofficially, we’ve found many of our existing models to *underfit* on this dataset so far. I think this is a great sign – suggesting that the breadth of the dataset exceeds prior data sources.

Major kudos to @SashaKhazatsky & @KarlPertsch, who coordinated the group. But, it was truly a team effort and really wouldn’t have been possible without everyone’s willingness to contribute.

Check out Sasha's thread for more details!

Dear Chelsea Finn, Could you please teach us more and deeply about robotics and RL, as an Student in India, we dont have anyone good to teach or view as good as you do.

@nathan___gage

@jvboid

People outside robotics may not realize how cool this is 👌