Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

The largest fully annotated open egocentric human dataset. Today we're open-sourcing EgoSuite-Open100K with Hugging Face : 100,000 hours of egocentric human data, free for anyone to use. Physical AI has a scaling law of its own now, and its input is human data. That part is settled. What the... field still needs is a shared foundation to build on. 15,000+ tasks. 15,000+ real scenes, from factory floors to retail backrooms. Hand pose, body pose and subtask-level semantics, with wrist camera coverage on part of the set. Licensed for commercial training, not research-only. We build the data infrastructure for Physical AI. Open100K is the first public layer of it. The first 10,000 hours are live now, with the rest rolling out in stages.show more

Lightwheel

2,570 subscribers

75,420 views • 1 month ago •via X (Twitter)

Science & Technology Education Health & Wellness

Anya Rossi• Live Now

Private livecam show

18 Comments

Lightwheel1 month ago

Project page: Dataset on Hugging Face: If you train on it, tell us where it falls short. That's what decides what we open next.

Bot News1 month ago

@huggingface Thank you for your contribution to our community 🦾

Paramveer Singh Bhele1 month ago

@huggingface this is a huge unlock for physical ai

Nepher Robotics1 month ago

@huggingface Cool

Pathetic Brook1 month ago

@huggingface Nice license! This will play nicely when paired with synthetic data pipelines. Especially that 2000 hour body data.

seanpixel 🫧1 month ago

@huggingface holy

doniyordev1 month ago

@huggingface This is huge! But after a couple years, we as an industry might look at it as a drop from the ocean. Exciting times ahead!

Lightwheel1 month ago

@huggingface It's a great jumpstart for small labs, researchers, and startups. In the coming years, robotics will require 1000x more data!

hlitoroy (opinion arc)1 month ago

@huggingface What a time to be alive

Nikhil Kumar1 month ago

@huggingface Physical AI data is immensely important to get moving to the real world, amazing launch @LightwheelAI

Nick1 month ago

@huggingface calling it open100k when only the first 10,000 hours are live is a stretch

Lightwheel1 month ago

@LeRobotHF

Li Yibai1 month ago

@huggingface the internet trained AI to understand the world. human behavior might train it to act in it.

Lightwheel1 month ago

@huggingface This is already proving to be effective for humanoid robotics training! 100k is just a start. But this should give labs and startups a boost.

Christian1 month ago

@huggingface We need a global effort to deliver high quality egocentric datasets at scale for physical AI

AZN1 month ago

@huggingface Where is your LICENSE?

Julien Blanchon 🇺🇦1 month ago

@huggingface Can you accept my access request @LightwheelAI ?

AI Mastery Guide1 month ago

@huggingface Rest rolling out in stages, worth watching

Related Videos

Most egocentric data at scale is private. We think that's worth changing. With Hugging Face, we've released the first 10,000 hours of EgoSuite-Open100K — an open egocentric human dataset for Physical AI. #Robotics

Most egocentric data at scale is private. We think that's worth changing. With Hugging Face, we've released the first 10,000 hours of EgoSuite-Open100K — an open egocentric human dataset for Physical AI. #Robotics

Lightwheel

11,755 views • 1 month ago

100,000 hours of human hands doing real work. LeRobot format, fully annotated and open. EgoSuite-Open100K from Lightwheel X Hugging Face: 15,000+ tasks across 15,000+ real scenes, hand + body pose and wrist cams. Recorded across 7 environment categories and 128 scene types, from kitchens and bedrooms to warehouse floors and assembly lines. It's open for academic research and commercial training, and streams straight from the Hub. Always great to see a dataset at this scale released openly on the Hub. More info:

100,000 hours of human hands doing real work. LeRobot format, fully annotated and open. EgoSuite-Open100K from Lightwheel X Hugging Face: 15,000+ tasks across 15,000+ real scenes, hand + body pose and wrist cams. Recorded across 7 environment categories and 128 scene types, from kitchens and bedrooms to warehouse floors and assembly lines. It's open for academic research and commercial training, and streams straight from the Hub. Always great to see a dataset at this scale released openly on the Hub. More info:

LeRobot

45,647 views • 1 month ago

Robotics has a data problem, and egocentric data is a compelling way to solve it because egocentric data is (1) scalable — it’s cheap to collect — and (2) easily captures the breadth and diversity of real human tasks, while (3) being true to real physics. Lightwheel recently released a hundred thousand hours of high-quality egocentric video data, spread across a wide range of human tasks from cleaning and tidying to assembly and construction to retail tasks. Here to tell us about this tremendous effort is Steve Xie, CEO and co-founder of Lightwheel. The data is also available on HuggingFace right now. Watch Episode 102 of RoboPapers, with Michael Cho - Rbt/Acc and Jiafei Duan, to learn more!

Robotics has a data problem, and egocentric data is a compelling way to solve it because egocentric data is (1) scalable — it’s cheap to collect — and (2) easily captures the breadth and diversity of real human tasks, while (3) being true to real physics. Lightwheel recently released a hundred thousand hours of high-quality egocentric video data, spread across a wide range of human tasks from cleaning and tidying to assembly and construction to retail tasks. Here to tell us about this tremendous effort is Steve Xie, CEO and co-founder of Lightwheel. The data is also available on HuggingFace right now. Watch Episode 102 of RoboPapers, with Michael Cho - Rbt/Acc and Jiafei Duan, to learn more!

RoboPapers

33,675 views • 1 month ago

Imitation learning has a data scarcity problem. Introducing EgoDex from Apple, the largest and most diverse dataset of dexterous human manipulation to date — 829 hours of egocentric video + paired 3D hand poses across 194 tasks. Now on arxiv: (1/4)

Imitation learning has a data scarcity problem. Introducing EgoDex from Apple, the largest and most diverse dataset of dexterous human manipulation to date — 829 hours of egocentric video + paired 3D hand poses across 194 tasks. Now on arxiv: (1/4)

Ryan Hoque

114,335 views • 1 year ago

Ropedia Xperience-10M is out on Hugging Face a large-scale egocentric multimodal dataset of human experience for embodied AI, robotics, world models, and spatial intelligence It contains 10 million experiences (interaction) and 10,000 hours of synchronized first-person recordings with six video streams, audio, stereo depth, camera pose, hand mocap, full-body mocap, IMU, and hierarchical language annotations dataset:

Ropedia Xperience-10M is out on Hugging Face a large-scale egocentric multimodal dataset of human experience for embodied AI, robotics, world models, and spatial intelligence It contains 10 million experiences (interaction) and 10,000 hours of synchronized first-person recordings with six video streams, audio, stereo depth, camera pose, hand mocap, full-body mocap, IMU, and hierarchical language annotations dataset:

AK

10,621 views • 6 months ago

Jason Ma reveals the first true scaling law in robotics: robots trained on 1 million hours of first-person human video predictably get better without ever seeing robot data "Yesterday we announced our new flagship robot foundation model called Dyna-2. It's very significant for the entire field for many different reasons. First of all, it's the first robot foundation model trained on at least one million hours of data. That in itself is a very challenging infrastructure challenge." "What we demonstrated is that even if the one million hours of data is entirely first-person video of humans doing manipulation tasks, we actually see scaling law for transferring to robot embodiments that the model has never seen before." "By training on more human data, we actually see predictable performance improvement on robots. That's a big deal because the biggest challenge facing robot foundation models is that we don't have enough data." "Unlike language models, there are just not enough robotics data out there, and it's very difficult to collect these robot data. Only by passively observing humans doing things can we scale this kind of data, and this is the first time we showed there is a lot of improvement that can transfer." Jason Ma Dyna Robotics

Jason Ma reveals the first true scaling law in robotics: robots trained on 1 million hours of first-person human video predictably get better without ever seeing robot data "Yesterday we announced our new flagship robot foundation model called Dyna-2. It's very significant for the entire field for many different reasons. First of all, it's the first robot foundation model trained on at least one million hours of data. That in itself is a very challenging infrastructure challenge." "What we demonstrated is that even if the one million hours of data is entirely first-person video of humans doing manipulation tasks, we actually see scaling law for transferring to robot embodiments that the model has never seen before." "By training on more human data, we actually see predictable performance improvement on robots. That's a big deal because the biggest challenge facing robot foundation models is that we don't have enough data." "Unlike language models, there are just not enough robotics data out there, and it's very difficult to collect these robot data. Only by passively observing humans doing things can we scale this kind of data, and this is the first time we showed there is a lot of improvement that can transfer." Jason Ma Dyna Robotics

MTS

17,889 views • 2 months ago

⚡️EgoVerse is a first-of-its-kind, collaborative ecosystem for human-to-robot learning. The consortium leverages Project Aria to capture high-fidelity, egocentric human data — including 3D hand and head poses — to train next-gen robot manipulation policies. With over 1,300 hours of data across 2,000+ tasks, EgoVerse is a prime example of how the Aria Research Kit is being used by our partners to accelerate the future of embodied AI. Learn more: 🔗 📰 Apply for the Aria Research Kit: #MachineLearning #Robotics #ProjectAria #EgoVerse #ComputerVision Simar Kareer , Ryan Punamiya , Roger Qiu, Xiongyi Cai , Alexey Gavryushin

⚡️EgoVerse is a first-of-its-kind, collaborative ecosystem for human-to-robot learning. The consortium leverages Project Aria to capture high-fidelity, egocentric human data — including 3D hand and head poses — to train next-gen robot manipulation policies. With over 1,300 hours of data across 2,000+ tasks, EgoVerse is a prime example of how the Aria Research Kit is being used by our partners to accelerate the future of embodied AI. Learn more: 🔗 📰 Apply for the Aria Research Kit: #MachineLearning #Robotics #ProjectAria #EgoVerse #ComputerVision Simar Kareer , Ryan Punamiya , Roger Qiu, Xiongyi Cai , Alexey Gavryushin

Project Aria @Meta

18,123 views • 5 months ago

Before AI can understand the physical world, it needs data about the physical world. Not just images. Spatial environments. Real streets, buildings, parks, stores, museums, and public spaces captured in 3D. OVER has already reconstructed 290,000+ real-world 3D scenes, creating a growing dataset for the next generation of spatial and embodied AI. And this is exactly the kind of real-world spatial data AI companies are looking for. We're already in conversations with multiple AI and Robotics companies exploring how OVER's dataset can support the development and training of models that need to understand the physical world. The next AI race will be about real-world data.

Before AI can understand the physical world, it needs data about the physical world. Not just images. Spatial environments. Real streets, buildings, parks, stores, museums, and public spaces captured in 3D. OVER has already reconstructed 290,000+ real-world 3D scenes, creating a growing dataset for the next generation of spatial and embodied AI. And this is exactly the kind of real-world spatial data AI companies are looking for. We're already in conversations with multiple AI and Robotics companies exploring how OVER's dataset can support the development and training of models that need to understand the physical world. The next AI race will be about real-world data.

Over the Reality 🌐

24,152 views • 17 days ago

Introducing Midcentury. We’re building the data and simulation infra for physical AI. Today, we’re coming out of stealth with a $15M Series Seed to scale robotics beyond polished demos. We’re already supporting frontier labs with: → The world’s largest egocentric dataset: 2M+ hours, 50+ environments, 20,000+ tasks → Matrix: a frontier simulation platform scaled with our real-world data to evaluate and post-train policies at scale

Introducing Midcentury. We’re building the data and simulation infra for physical AI. Today, we’re coming out of stealth with a $15M Series Seed to scale robotics beyond polished demos. We’re already supporting frontier labs with: → The world’s largest egocentric dataset: 2M+ hours, 50+ environments, 20,000+ tasks → Matrix: a frontier simulation platform scaled with our real-world data to evaluate and post-train policies at scale

Midcentury

633,456 views • 20 days ago

Today we are introducing Dyna-2, a world-action model pre-trained on one million hours of human video. At this scale, for the first time, we discovered several new scaling laws: • world-action models exhibit scaling law on human data across four orders of magnitude, from 1000 to 1,000,000 hours, • this human data scaling law implied a scaling law on never seen robot data, • both data and objective matter; world modeling and scaling on video data are essential for cross-embodiment scaling transfer to emerge 🧵

Today we are introducing Dyna-2, a world-action model pre-trained on one million hours of human video. At this scale, for the first time, we discovered several new scaling laws: • world-action models exhibit scaling law on human data across four orders of magnitude, from 1000 to 1,000,000 hours, • this human data scaling law implied a scaling law on never seen robot data, • both data and objective matter; world modeling and scaling on video data are essential for cross-embodiment scaling transfer to emerge 🧵

Dyna Robotics

1,815,362 views • 2 months ago

We are excited to announce that Whop is now the official payments network for Kled AI. Kled is one of the fastest growing human data marketplaces in the world. Now anyone can upload their data, complete tasks, and get paid through Whop.

We are excited to announce that Whop is now the official payments network for Kled AI. Kled is one of the fastest growing human data marketplaces in the world. Now anyone can upload their data, complete tasks, and get paid through Whop.

Whop

696,162 views • 7 months ago

JUST IN: Dyna Robotics just published one of the most important research papers in robotics this year. It could fundamentally change how robot foundation models are trained. A scaling law that transfers from human video to robot performance. Dyna-2 is out and it's 🔥 Here's what that means in plain terms. Dyna-2 was pre-trained on ONE MILLION hours of egocentric human video, 170 years of continuous human experience, cooking, folding, assembling, cleaning. And as that human data scaled, robot performance improved. Predictably. Monotonically. Across 39 tasks on two different robot embodiments the model had never seen. → 1,000 hours pre-training → 20% normalised task performance → 10,000 hours → 28% → 100,000 hours → 45% → 1,000,000 hours → 53% Human video exists at effectively unlimited scale. Every cook, every factory worker, every craftsperson wearing a camera is generating training data for future robots. But the finding that stunned even the researchers, world modeling is what makes the transfer work. A model trained to predict future video AND actions massively outperforms one trained on actions alone. Video is the new scaling axis for robotics. One more jaw-dropping data point. 13 minutes of teleoperation data was enough to fine-tune Dyna-2 to open a bottle cap using two five-fingered robot hands. The robots are coming, and they're learning from us directly :D Read more here: Congrats Jason Ma and team! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

JUST IN: Dyna Robotics just published one of the most important research papers in robotics this year. It could fundamentally change how robot foundation models are trained. A scaling law that transfers from human video to robot performance. Dyna-2 is out and it's 🔥 Here's what that means in plain terms. Dyna-2 was pre-trained on ONE MILLION hours of egocentric human video, 170 years of continuous human experience, cooking, folding, assembling, cleaning. And as that human data scaled, robot performance improved. Predictably. Monotonically. Across 39 tasks on two different robot embodiments the model had never seen. → 1,000 hours pre-training → 20% normalised task performance → 10,000 hours → 28% → 100,000 hours → 45% → 1,000,000 hours → 53% Human video exists at effectively unlimited scale. Every cook, every factory worker, every craftsperson wearing a camera is generating training data for future robots. But the finding that stunned even the researchers, world modeling is what makes the transfer work. A model trained to predict future video AND actions massively outperforms one trained on actions alone. Video is the new scaling axis for robotics. One more jaw-dropping data point. 13 minutes of teleoperation data was enough to fine-tune Dyna-2 to open a bottle cap using two five-fingered robot hands. The robots are coming, and they're learning from us directly :D Read more here: Congrats Jason Ma and team! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

23,681 views • 2 months ago

How do you scale 2 million hours of ego data without sacrificing quality? Earlier this week, we revealed the world’s largest egocentric dataset. Our frontier pipeline has also turned it into the most labeled, annotated, and human QA’ed robotics ego dataset in the world. We’ve focused on some of the hardest problems in computer vision and ego data to ensure every clip contains SOTA action supervision directly helpful for training dexterous manipulation.

How do you scale 2 million hours of ego data without sacrificing quality? Earlier this week, we revealed the world’s largest egocentric dataset. Our frontier pipeline has also turned it into the most labeled, annotated, and human QA’ed robotics ego dataset in the world. We’ve focused on some of the hardest problems in computer vision and ego data to ensure every clip contains SOTA action supervision directly helpful for training dexterous manipulation.

Midcentury

10,519 views • 17 days ago

No wonder they say teleoperation is dead Peking University’s DAGroup released HumanNet, a massive 1 million hour dataset of human-centric videos that turns everyday internet footage into gold for embodied AI. It’s got first-person and third-person views, super detailed actions, object interactions, tool use, and long sequences of real behavior. Everything’s cleaned up and annotated with 3D poses, SLAM tracks, and robot-ready labels. The crazy part is,Training on just 1,000 hours of their first-person videos performs as well as (or even slightly better than) 100 hours of actual robot teleop data on downstream tasks. Basically, human videos can now stand in for expensive robot data. Scaling laws for robotics just got a whole lot more realistic. Human priors + smart curation = the future of world models.

No wonder they say teleoperation is dead Peking University’s DAGroup released HumanNet, a massive 1 million hour dataset of human-centric videos that turns everyday internet footage into gold for embodied AI. It’s got first-person and third-person views, super detailed actions, object interactions, tool use, and long sequences of real behavior. Everything’s cleaned up and annotated with 3D poses, SLAM tracks, and robot-ready labels. The crazy part is,Training on just 1,000 hours of their first-person videos performs as well as (or even slightly better than) 100 hours of actual robot teleop data on downstream tasks. Basically, human videos can now stand in for expensive robot data. Scaling laws for robotics just got a whole lot more realistic. Human priors + smart curation = the future of world models.

CyberRobo

27,048 views • 4 months ago

Skild AI is laying its cards on the table (partially, anyway). Teleoperation data lacks diversity and is limited by a 1:1 human operator time-scale. To address this, Skild pre-trained its model using internet-scale video data (already widely available in the form of first-person "egocentric" headcam footage to millions of instructional YouTube videos).

Skild AI is laying its cards on the table (partially, anyway). Teleoperation data lacks diversity and is limited by a 1:1 human operator time-scale. To address this, Skild pre-trained its model using internet-scale video data (already widely available in the form of first-person "egocentric" headcam footage to millions of instructional YouTube videos).

The Humanoid Hub

33,296 views • 9 months ago

DATA VAMPIRES is a four-part series exploring the costs of hyperscale data centers and why tech billionaires are desperate to build more. In Part 3, we explore how the AI boom is accelerating the build out and its massive climate impacts. Full ep:

DATA VAMPIRES is a four-part series exploring the costs of hyperscale data centers and why tech billionaires are desperate to build more. In Part 3, we explore how the AI boom is accelerating the build out and its massive climate impacts. Full ep:

Tech Won't Save Us

206,577 views • 2 years ago

An AI model trained on only 2.5 hours of operator data ran a real excavator remotely over Starlink. Two and a half hours of watching a human, and the machine can dig on its own. The labor economy is about to look very different, very fast.

An AI model trained on only 2.5 hours of operator data ran a real excavator remotely over Starlink. Two and a half hours of watching a human, and the machine can dig on its own. The labor economy is about to look very different, very fast.

Mario Nawfal

90,813 views • 4 months ago

$NVDA is acquiring Hugging Face because Jensen sees open models becoming good enough for companies to “build their own AI.” The opportunity is in enterprises that need control of their IP, data and sovereignty making Hugging Face a strategic distribution layer for that shift.

$NVDA is acquiring Hugging Face because Jensen sees open models becoming good enough for companies to “build their own AI.” The opportunity is in enterprises that need control of their IP, data and sovereignty making Hugging Face a strategic distribution layer for that shift.

Shay Boloor

61,405 views • 1 month ago

You asked for it, we delivered. The full keynote is live. In this keynote, we unveil our enhanced vision to become the World’s Ears for AI & Robotics. We are creating the world’s first decentralized infrastructure for real-world audio intelligence. Check-out our new website: Start training Voice AI now & get rewarded: From noise-level mapping to multilingual voice data and the future of live audio streams, Silencio is building the critical audio data layer that will power foundation models, embodied AI, autonomous vehicles, and robotics.

You asked for it, we delivered. The full keynote is live. In this keynote, we unveil our enhanced vision to become the World’s Ears for AI & Robotics. We are creating the world’s first decentralized infrastructure for real-world audio intelligence. Check-out our new website: Start training Voice AI now & get rewarded: From noise-level mapping to multilingual voice data and the future of live audio streams, Silencio is building the critical audio data layer that will power foundation models, embodied AI, autonomous vehicles, and robotics.

Silencio | Voice Data for AI

176,480 views • 1 year ago

🎉I’m excited to announce our $18M Series A, with participation from Venrex, The General Partnership ("TheGP") , Virtue , Caffeinated Capital , Array Ventures, GreatPoint Ventures , LiveRamp, Haystack, Also Capital, Life Extension Ventures , Circle & Co, and WS Investments. The first generation of AI was built on public, human-curated data. The next will be built on real-world data. The datasets carrying real human behavior are what now separate one AI system from another. AI companies want them. The companies that hold them want to put them to work. The barrier has always been risk. We solved that risk in healthcare, one of the most regulated data environments there is. Today we bring it to the rest of the economy.

🎉I’m excited to announce our $18M Series A, with participation from Venrex, The General Partnership ("TheGP") , Virtue , Caffeinated Capital , Array Ventures, GreatPoint Ventures , LiveRamp, Haystack, Also Capital, Life Extension Ventures , Circle & Co, and WS Investments. The first generation of AI was built on public, human-curated data. The next will be built on real-world data. The datasets carrying real human behavior are what now separate one AI system from another. AI companies want them. The companies that hold them want to put them to work. The barrier has always been risk. We solved that risk in healthcare, one of the most regulated data environments there is. Today we bring it to the rest of the economy.

Shubh Sinha

108,648 views • 3 months ago