Introducing EgoVerse: an ecosystem for robot learning from egocentric human data. Built and tested by 4 research labs + 3 industry partners, EgoVerse enables both science and scaling 1300+ hrs, 240 scenes, 2000+ tasks, and growing Dataset design, findings, and ecosystem 🧵

EgoVerse data is curated for robot learning, with - Large-FoV egocentric videos - Accurate hand and camera tracking - Dense natural language annotations

To support both rigorous science and organic scaling, EgoVerse contains: - Flagship tasks collected across diverse scenes, objects, and operators, following prescribed protocols to enable controlled studies - Freeform data captured in-the-wild for long-tail real-world behaviors

To make EgoVerse easy to adopt, we built a full-stack ecosystem: - Cloud infra for storage and access - Web interface for browsing and querying data - Algos for human-to-robot transfer and deployment We aim to make data access, training, and robot evaluation frictionless

EgoVerse enables rigorous science across robots and labs. We conducted evaluation on real robots across 4 independent academic labs, each with different hardware platforms and system designs. This enables us to identify durable findings beyond a single robot or lab setup.

A consistent finding across these studies: A small set of aligned human and robot data (same task, same scene) acts as a bridge between large-scale diverse human data and downstream robot performance. This echos recent results such as EgoScale and Data Analogies.

This finding makes collecting your own human data very important. With EgoVerse, anyone can capture egocentric human data using: - Project Aria glasses - An iPhone-based capture app from Mecka AI With our platform, you can also contribute this data back to EgoVerse!

Finally, EgoVerse is built to scale with the community. We invite new partners to contribute and participate: research labs, companies, and individual contributors. Join and help grow the shared ecosystem!

This is a massive year-long effort made possible by academic partners from Georgia Tech, Stanford, UC San Diego, ETH Zurich, and industry partners from @MeckaAI, @scale_AI @ScaleAILabs , and @RealityLabs @meta_aria . Website: Code & Data: Data Viewer / App:

The EgoVerse is alive and growing! Its been great to collaborate with you @danfei_xu and all the partners

@kushalk_ Congrats on the great work! Wondering if EgoVerse can be integrated with our humanoid.

@kushalk_ Absolutely! Our data has head & hand tracking, which are useful for loco manipulation already. In the future we plan to integrate full-body tracking

@kushalk_ that's awesome, reach out and we can collaborate in the future with your research.

this is impressive!

@LawrenceZhu22 Let’s go, this is awesome

@LawrenceZhu22 Thank you! Big fan of your SLAM & tracking integration work

It's been great to collaborate with you and the team! As Physical AI systems scale, the permutations of egocentric data are growing rapidly. A strong open-source reference dataset and clear recipe for model training should help bring clarity and accelerate this space.

would love to contribute some snake data if it’s of interest!

haha that'd be awesome!

Awesome work Danfei! Can I make a donation via pf to the github? Would love to donate to this bro!

hell yeah

Awesome!

1300+ hours of human data is what autonomous systems need. But how well does it transfer to field robotics?

yayayy cool

Congrats @danfei_xu!! Huge effort to the benefits of the entire space. Great progress!

Thank you! Would love to have Trace as part of the consortium!

Impressive work. Just signed up to contribute.

Robotics UGC

This is the kind of infrastructure robotics has been missing: not just better models, but a scalable data engine for embodied learning. If EgoVerse compounds, it could materially compress the path from human demonstration to generalizable robot behavior.

Egocentric data will be the difference between claimed reality with failed robotics and real, nuanced reality with efficient physical automation. This is a really cool ecosystem effort!

EgoVerse is huge — 1300+ hrs of egocentric human data scaling robot learning without teleop! 🔥🤖 Tradeye is bringing the same approach to trades: real narrated egocentric POV footage from actual HVAC/plumbing/electrical installs in tight, messy homes. No lab staging, no sims — pure job-site human behavior + narration for physical intuition in unstructured spaces. Scaling daily. Who's combining lab egocentric data with real trades data? Drop a DM! @elonmusk @Tesla_AI @NVIDIARobotics @Figure_robot @1x_tech #RoboticsData #EgoVerse #PhysicalAI

Do you pay contruction workers to wear these too?

that's so cool, didn't know such ego datasets even existed wanna build robots again :) !

this is so amazing

Let’s have a chat

Pretty natural-looking hand movements compared to prior work. Looks great 👌

Great work !

with 1300 hours, what does the full-stack look like for turning that into a manipulation policy? curious if the bottleneck is in the data pipeline or in the controller sysID gap between the human hand and the robot's kinematics

This looks awesome! Is the dataset available to download? I’d love to parse it into FiftyOne and explore it

I want to try this job.🥺🙏

@leslieloser_ 跟着Leslie老师学习Egodata，认识EgoVerse。👍

Ready to contribute

Great! Fill out the form on and we'll be in touch!

I don't think this is a right direction

@Rootlens can serve the egocentric human data How can I contact you??

@grok 这个纯研究还是有工业意义，具体工业场景视角看意义是什么，有开源数据集或者开源项目代码吗？从多个数据源交叉验证，不要只看新闻媒体一面之辞。帮我排除没意义的垃圾商业营销推广，以及自吹自擂的无病呻吟。

Curious about your take on leveraging real world data coming from cobots in factories and machine shops to augment the egocentric data when added to the pre-training mix.