Introducing EgoVerse: an ecosystem for robot learning from egocentric human data.
312.656 görüntüleme
46 Yorum
Yorumlar X (Twitter) üzerindeki orijinal gönderiden alınmıştır

@danfei_xu
EgoVerse data is curated for robot learning, with - Large-FoV egocentric videos - Accurate hand and camera tracking - Dense natural language annotations
80 beğeni

@danfei_xu
To support both rigorous science and organic scaling, EgoVerse contains: - Flagship tasks collected across diverse scenes, objects, and operators, following prescribed protocols to enable controlled studies - Freeform data captured in-the-wild for long-tail real-world behaviors
36 beğeni

@danfei_xu
To make EgoVerse easy to adopt, we built a full-stack ecosystem: - Cloud infra for storage and access - Web interface for browsing and querying data - Algos for human-to-robot transfer and deployment We aim to make data access, training, and robot evaluation frictionless
29 beğeni

@danfei_xu
EgoVerse enables rigorous science across robots and labs. We conducted evaluation on real robots across 4 independent academic labs, each with different hardware platforms and system designs. This enables us to identify durable findings beyond a single robot or lab setup.
35 beğeni

@danfei_xu
A consistent finding across these studies: A small set of aligned human and robot data (same task, same scene) acts as a bridge between large-scale diverse human data and downstream robot performance. This echos recent results such as EgoScale and Data Analogies.
19 beğeni

@danfei_xu
This finding makes collecting your own human data very important. With EgoVerse, anyone can capture egocentric human data using: - Project Aria glasses - An iPhone-based capture app from Mecka AI With our platform, you can also contribute this data back to EgoVerse!
30 beğeni

@danfei_xu
Finally, EgoVerse is built to scale with the community. We invite new partners to contribute and participate: research labs, companies, and individual contributors. Join and help grow the shared ecosystem!
22 beğeni

@danfei_xu
This is a massive year-long effort made possible by academic partners from Georgia Tech, Stanford, UC San Diego, ETH Zurich, and industry partners from @MeckaAI, @scale_AI @ScaleAILabs , and @RealityLabs @meta_aria . Website: Code & Data: Data Viewer / App:
87 beğeni

@joshavata
The EgoVerse is alive and growing! Its been great to collaborate with you @danfei_xu and all the partners
6 beğeni

@archerlin__
@kushalk_ Congrats on the great work! Wondering if EgoVerse can be integrated with our humanoid.
6 beğeni

@danfei_xu
@kushalk_ Absolutely! Our data has head & hand tracking, which are useful for loco manipulation already. In the future we plan to integrate full-body tracking
2 beğeni

@archerlin__
@kushalk_ that's awesome, reach out and we can collaborate in the future with your research.

@zhaohang0124
this is impressive!
5 beğeni

@pablovelagomez1
@LawrenceZhu22 Let’s go, this is awesome
5 beğeni

@danfei_xu
@LawrenceZhu22 Thank you! Big fan of your SLAM & tracking integration work
2 beğeni

@benlevin
It's been great to collaborate with you and the team! As Physical AI systems scale, the permutations of egocentric data are growing rapidly. A strong open-source reference dataset and clear recipe for model training should help bring clarity and accelerate this space.
5 beğeni

@tejessrivalsan
would love to contribute some snake data if it’s of interest!
5 beğeni

@danfei_xu
haha that'd be awesome!
1 beğeni

@imhimgoat01
Awesome work Danfei! Can I make a donation via pf to the github? Would love to donate to this bro!
4 beğeni

@k7agar
hell yeah
4 beğeni























