Gezinmeyi atla
Introducing EgoVerse: an ecosystem for robot learning from egocentric human data.
0:00 / 0:59

Introducing EgoVerse: an ecosystem for robot learning from egocentric human data.

Danfei Xu

@danfei_xu14 B takipçi

X profili
907 beğeniİndirOrijinal gönderi

312.656 görüntüleme

Introducing EgoVerse: an ecosystem for robot learning from egocentric human data. Built and tested by 4 research labs + 3 industry partners, EgoVerse enables both science and scaling 1300+ hrs, 240 scenes, 2000+ tasks, and growing Dataset design, findings, and ecosystem 🧵

46 Yorum

Yorumlar X (Twitter) üzerindeki orijinal gönderiden alınmıştır

  • Danfei Xu profil fotoğrafı

    @danfei_xu

    EgoVerse data is curated for robot learning, with - Large-FoV egocentric videos - Accurate hand and camera tracking - Dense natural language annotations

    80 beğeni

  • Danfei Xu profil fotoğrafı

    @danfei_xu

    To support both rigorous science and organic scaling, EgoVerse contains: - Flagship tasks collected across diverse scenes, objects, and operators, following prescribed protocols to enable controlled studies - Freeform data captured in-the-wild for long-tail real-world behaviors

    36 beğeni

  • Danfei Xu profil fotoğrafı

    @danfei_xu

    To make EgoVerse easy to adopt, we built a full-stack ecosystem: - Cloud infra for storage and access - Web interface for browsing and querying data - Algos for human-to-robot transfer and deployment We aim to make data access, training, and robot evaluation frictionless

    29 beğeni

  • Danfei Xu profil fotoğrafı

    @danfei_xu

    EgoVerse enables rigorous science across robots and labs. We conducted evaluation on real robots across 4 independent academic labs, each with different hardware platforms and system designs. This enables us to identify durable findings beyond a single robot or lab setup.

    35 beğeni

  • Danfei Xu profil fotoğrafı

    @danfei_xu

    A consistent finding across these studies: A small set of aligned human and robot data (same task, same scene) acts as a bridge between large-scale diverse human data and downstream robot performance. This echos recent results such as EgoScale and Data Analogies.

    19 beğeni

  • Danfei Xu profil fotoğrafı

    @danfei_xu

    This finding makes collecting your own human data very important. With EgoVerse, anyone can capture egocentric human data using: - Project Aria glasses - An iPhone-based capture app from Mecka AI With our platform, you can also contribute this data back to EgoVerse!

    30 beğeni

  • Danfei Xu profil fotoğrafı

    @danfei_xu

    Finally, EgoVerse is built to scale with the community. We invite new partners to contribute and participate: research labs, companies, and individual contributors. Join and help grow the shared ecosystem!

    22 beğeni

  • Danfei Xu profil fotoğrafı

    @danfei_xu

    This is a massive year-long effort made possible by academic partners from Georgia Tech, Stanford, UC San Diego, ETH Zurich, and industry partners from @MeckaAI, @scale_AI @ScaleAILabs , and @RealityLabs @meta_aria . Website: Code & Data: Data Viewer / App:

    87 beğeni

  • Josh profil fotoğrafı

    @joshavata

    The EgoVerse is alive and growing! Its been great to collaborate with you @danfei_xu and all the partners

    6 beğeni

  • Archer Lin profil fotoğrafı

    @archerlin__

    @kushalk_ Congrats on the great work! Wondering if EgoVerse can be integrated with our humanoid.

    6 beğeni

  • Danfei Xu profil fotoğrafı

    @danfei_xu

    @kushalk_ Absolutely! Our data has head & hand tracking, which are useful for loco manipulation already. In the future we plan to integrate full-body tracking

    2 beğeni

  • Archer Lin profil fotoğrafı

    @archerlin__

    @kushalk_ that's awesome, reach out and we can collaborate in the future with your research.

  • Hang Zhao profil fotoğrafı

    @zhaohang0124

    this is impressive!

    5 beğeni

  • Pablo Vela profil fotoğrafı

    @pablovelagomez1

    @LawrenceZhu22 Let’s go, this is awesome

    5 beğeni

  • Danfei Xu profil fotoğrafı

    @danfei_xu

    @LawrenceZhu22 Thank you! Big fan of your SLAM & tracking integration work

    2 beğeni

  • Benjamin Levin profil fotoğrafı

    @benlevin

    It's been great to collaborate with you and the team! As Physical AI systems scale, the permutations of egocentric data are growing rapidly. A strong open-source reference dataset and clear recipe for model training should help bring clarity and accelerate this space.

    5 beğeni

  • Tejes Srivalsan profil fotoğrafı

    @tejessrivalsan

    would love to contribute some snake data if it’s of interest!

    5 beğeni

  • Danfei Xu profil fotoğrafı

    @danfei_xu

    haha that'd be awesome!

    1 beğeni

  • Fade profil fotoğrafı

    @imhimgoat01

    Awesome work Danfei! Can I make a donation via pf to the github? Would love to donate to this bro!

    4 beğeni

  • atharva ☆ profil fotoğrafı

    @k7agar

    hell yeah

    4 beğeni