Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

What does a real humanoid robotics dataset actually look like? Humanoid robots are expensive and complex, which makes their data unusually difficult to collect - and difficult for most people to get hands-on experience with. We previously shared a post on conversion code for the new Human in the...

rerundotio's profile picture

Rerun

7,596 subscribers

11,355 görüntüleme • 1 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Eastworlds aims to build the largest dataset of high-quality humanoid teleop data outside of China. While we believe human egocentric data will play a big role in pretraining embodied AI, robot teleop data remains the closest thing to expert demonstration.
0:11

Eastworlds aims to build the largest dataset of high-quality humanoid teleop data outside of China. While we believe human egocentric data will play a big role in pretraining embodied AI, robot teleop data remains the closest thing to expert demonstration.

Eastworlds

62,755 görüntüleme • 5 ay önce

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:
0:18

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:

Chenfeng_X

28,363 görüntüleme • 2 yıl önce

The largest open-source humanoid teleop dataset, 10+ TB collected from real homes, is now available to everyone. Proud to support BitRobot 🦾's journey in pushing real robotics to the frontier at Virtuals Protocol.
0:14

The largest open-source humanoid teleop dataset, 10+ TB collected from real homes, is now available to everyone. Proud to support BitRobot 🦾's journey in pushing real robotics to the frontier at Virtuals Protocol.

Virtuals Protocol

30,932 görüntüleme • 4 ay önce

Unlike LLMs, we can’t scrape the internet for robot data Announcing Project Go-Big: we’re building the world's largest humanoid pretraining dataset This is accelerated by our partnership with Brookfield, who owns over 100,000 residential units
0:11

Unlike LLMs, we can’t scrape the internet for robot data Announcing Project Go-Big: we’re building the world's largest humanoid pretraining dataset This is accelerated by our partnership with Brookfield, who owns over 100,000 residential units

Figure

616,933 görüntüleme • 1 yıl önce

Turns out you can train humanoid hands without any robot data. The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.
0:18

Turns out you can train humanoid hands without any robot data. The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.

Lerrel Pinto

35,012 görüntüleme • 4 ay önce

Spatial AI ( is building large real-world datasets to teach robots how to navigate the world and complete tasks. Their first open source dataset, SEA (Spatial Everyday Activities), is the largest curated egocentric dataset of people carrying out real tasks, with 10,000 hours of data.
0:11

Spatial AI ( is building large real-world datasets to teach robots how to navigate the world and complete tasks. Their first open source dataset, SEA (Spatial Everyday Activities), is the largest curated egocentric dataset of people carrying out real tasks, with 10,000 hours of data.

Y Combinator

20,944 görüntüleme • 11 ay önce

Qubic #OpenScience #Neuraxon Life 2.5 (Research) version code released & and a new Hugging Face massive dataset, almost 500Gb of raw TimeSeries Game data, 10 Sessions with 270+ rounds each, in total 100K+ Nxers in 2791 games with 3.76 Million Neurons & 17M Synapses to explore in a huge 1 Billion + events. By Jose Sánchez David Vivancos - e/acc for Qubic Science: Open Data Source Code: Dataset: Paper : Stay Tunned for more releases this week!
0:10

Qubic #OpenScience #Neuraxon Life 2.5 (Research) version code released & and a new Hugging Face massive dataset, almost 500Gb of raw TimeSeries Game data, 10 Sessions with 270+ rounds each, in total 100K+ Nxers in 2791 games with 3.76 Million Neurons & 17M Synapses to explore in a huge 1 Billion + events. By Jose Sánchez David Vivancos - e/acc for Qubic Science: Open Data Source Code: Dataset: Paper : Stay Tunned for more releases this week!

David Vivancos - e/acc

12,663 görüntüleme • 9 ay önce

Announcing our commercial partnership with Booster Robotics Booster builds humanoid robot hardware, OS, and developer tools to make humanoid robots more affordable, reliable, and practical. The partnership centers on using simulation to multiply the value of real-robot data—expanding teleoperation demonstrations into scalable training data across diverse tasks and scenes. This joint effort powers sim-real co-training and foundation-model development, accelerating progress from hardware iteration to deployable robot policies. Together, we're building simulation‑powered data infrastructure for Physical AI — making scalable training data accessible to model developers and the broader robotics ecosystem.
0:12

Announcing our commercial partnership with Booster Robotics Booster builds humanoid robot hardware, OS, and developer tools to make humanoid robots more affordable, reliable, and practical. The partnership centers on using simulation to multiply the value of real-robot data—expanding teleoperation demonstrations into scalable training data across diverse tasks and scenes. This joint effort powers sim-real co-training and foundation-model development, accelerating progress from hardware iteration to deployable robot policies. Together, we're building simulation‑powered data infrastructure for Physical AI — making scalable training data accessible to model developers and the broader robotics ecosystem.

Axis Robotics

72,872 görüntüleme • 2 ay önce

Let's skip the metal humanoid robots for a moment. Meet ultra-realistic humanoid robot friend, Aria, created by the sex robot company Realbotix. You can talk to her in real time, and she can even respond with emojis. These capabilities are beginning to spread to companionship, psychological therapy, and medical rehabilitation for elderly people living alone.
0:14

Let's skip the metal humanoid robots for a moment. Meet ultra-realistic humanoid robot friend, Aria, created by the sex robot company Realbotix. You can talk to her in real time, and she can even respond with emojis. These capabilities are beginning to spread to companionship, psychological therapy, and medical rehabilitation for elderly people living alone.

CyberRobo

92,044 görüntüleme • 1 yıl önce

NVIDIA announces the first open humanoid robot reference design built for robotics research. The NVIDIA Isaac GR00T Reference Humanoid Robot combines the Unitree H2 humanoid robot, Sharpa Wave five-fingered hands for dexterous manipulation, Jetson Thor onboard compute, and Isaac GR00T open software and models, giving researchers a full-stack platform from data capture to model deployment. Read the #NVIDIAGTC Taipei announcement:
0:20

NVIDIA announces the first open humanoid robot reference design built for robotics research. The NVIDIA Isaac GR00T Reference Humanoid Robot combines the Unitree H2 humanoid robot, Sharpa Wave five-fingered hands for dexterous manipulation, Jetson Thor onboard compute, and Isaac GR00T open software and models, giving researchers a full-stack platform from data capture to model deployment. Read the #NVIDIAGTC Taipei announcement:

NVIDIA Robotics

175,453 görüntüleme • 5 ay önce

Pi Ventures-Backed OpenMind Opens Waitlist For Humanoid Robot Control Pi Network (Pi Network) Ventures backed OpenMind (OpenMind) has opened the waitlist for Instabody, a platform that lets users remotely control a real humanoid robot. The company said users can see the world through the robot's eyes and directly control its movements. It believes giving people the controls will help shape how robots evolve. OpenMind previously launched a robot app store built on its OM1 operating system, allowing developers to add new skills to humanoid and quadruped robots.
0:18

Pi Ventures-Backed OpenMind Opens Waitlist For Humanoid Robot Control Pi Network (Pi Network) Ventures backed OpenMind (OpenMind) has opened the waitlist for Instabody, a platform that lets users remotely control a real humanoid robot. The company said users can see the world through the robot's eyes and directly control its movements. It believes giving people the controls will help shape how robots evolve. OpenMind previously launched a robot app store built on its OM1 operating system, allowing developers to add new skills to humanoid and quadruped robots.

BSCN

56,252 görüntüleme • 3 ay önce

And we’re live!! 🚀🚀🚀 We got OpenAI to generate graphs 📊 for any dataset 🤯 This week Antony and I hacked together a free tool to chat with your data goodbye complex pivot tables, here's the story 👇
0:18

And we’re live!! 🚀🚀🚀 We got OpenAI to generate graphs 📊 for any dataset 🤯 This week Antony and I hacked together a free tool to chat with your data goodbye complex pivot tables, here's the story 👇

Dominic Whyte

459,729 görüntüleme • 3 yıl önce

We just hit a new huge milestone. 🏆 Thanks to our amazing community, we have now reached 170,000 3D Maps across the world. Here are the numbers that define what we’ve built so far: ↳ 170,000 mapped locations ↳ 767 TB of world data ↳ 85.2M+ images captured ↳ And a steady, unstoppable month over month growth. A single dataset designed to help machines see, understand, and navigate space, also used to train spatial AI and visual models, and to geolocalize robots and experiences with precision through our VPS, in order to interact seamlessly with the real world. And yes... you read that right! We are already in talks with several top AI companies that want access to our dataset to train their AI models and to support what comes next, including robotics and advanced visual and video models. It is all one story, one ecosystem, one vision, and it is called Over the Reality.
0:10

We just hit a new huge milestone. 🏆 Thanks to our amazing community, we have now reached 170,000 3D Maps across the world. Here are the numbers that define what we’ve built so far: ↳ 170,000 mapped locations ↳ 767 TB of world data ↳ 85.2M+ images captured ↳ And a steady, unstoppable month over month growth. A single dataset designed to help machines see, understand, and navigate space, also used to train spatial AI and visual models, and to geolocalize robots and experiences with precision through our VPS, in order to interact seamlessly with the real world. And yes... you read that right! We are already in talks with several top AI companies that want access to our dataset to train their AI models and to support what comes next, including robotics and advanced visual and video models. It is all one story, one ecosystem, one vision, and it is called Over the Reality.

Over the Reality 🌐

93,237 görüntüleme • 11 ay önce

It's 2030 and you are reviewing humanoid robots. A Tesla. A Google. An Apple. An OpenAI. A Meta. A Figure. And a bunch of Chinese-made ones. Which one is best, and why? I think the Tesla understands the world much better. Why? There were eight Teslas around me on the freeway today. Start there. No other robot company has that data. But my robot is parked at the local high school twice a day. Its cameras see humans in all of our weirdness. How we move. Where we go. Where we walk. Who we talk with. What you are wearing. Whether your hair was combed this morning. That data will lead to robotics breakthroughs. Apple might keep up with its Vision Pro data, but it is too freaked out by the privacy implications of using said data. (On the front are six cameras and a couple of TOF -- Time Of Flight -- sensors that can see everything in your home in great detail). Google has a lot of data, for sure. All my: 1. Email. 2. Calendars. 3. Photos. 4. TV watching behavior. 5. Contacts. 6. Documents and spreadsheets. 7. Files. 8. Location data. So I expect Google's robot will be attractive to many. But how do you see the others shake out over the next five years? Make some guesses. But remember what an AI pioneer told me years ago about AI: it's all about the data. The Chinese ones have huge advantages: the Chinese have more data on their citizens, and many more citizens to boot AND they can make robots cheaper than we can. But now that you know OpenAI is building its own robot you have caught wind of what I've heard from many in San Francisco and Silicon Valley: that humanoid robots are the real prize of AI and will be highly profitable for those that can make them and find customers willing to buy them. Here, too, I learned long ago never to bet against Elon Musk. Will you?
0:16

It's 2030 and you are reviewing humanoid robots. A Tesla. A Google. An Apple. An OpenAI. A Meta. A Figure. And a bunch of Chinese-made ones. Which one is best, and why? I think the Tesla understands the world much better. Why? There were eight Teslas around me on the freeway today. Start there. No other robot company has that data. But my robot is parked at the local high school twice a day. Its cameras see humans in all of our weirdness. How we move. Where we go. Where we walk. Who we talk with. What you are wearing. Whether your hair was combed this morning. That data will lead to robotics breakthroughs. Apple might keep up with its Vision Pro data, but it is too freaked out by the privacy implications of using said data. (On the front are six cameras and a couple of TOF -- Time Of Flight -- sensors that can see everything in your home in great detail). Google has a lot of data, for sure. All my: 1. Email. 2. Calendars. 3. Photos. 4. TV watching behavior. 5. Contacts. 6. Documents and spreadsheets. 7. Files. 8. Location data. So I expect Google's robot will be attractive to many. But how do you see the others shake out over the next five years? Make some guesses. But remember what an AI pioneer told me years ago about AI: it's all about the data. The Chinese ones have huge advantages: the Chinese have more data on their citizens, and many more citizens to boot AND they can make robots cheaper than we can. But now that you know OpenAI is building its own robot you have caught wind of what I've heard from many in San Francisco and Silicon Valley: that humanoid robots are the real prize of AI and will be highly profitable for those that can make them and find customers willing to buy them. Here, too, I learned long ago never to bet against Elon Musk. Will you?

Robert Scoble

33,804 görüntüleme • 1 yıl önce

To replace animal testing with AI, we need MASSIVE human datasets. Today, we're thrilled to share Axiom's new data exploration tool, providing the ability to visually explore the world's largest primary human liver toxicity dataset. Built with Axiom's proprietary wetlab protocols, our dataset includes detailed liver toxicity profiles for over 100,000 distinct molecules. The key to this dataset is our ability to do high-throughput, multiplexed high-content screening with primary human liver cells. Traditionally, toxicity assays either sacrifice throughput or sacrifice biological relevance (using easy-to-grow immortalized cell lines instead of real human cells). We managed to combine throughput, physiological relevance, and multiplexing in one platform. The assays run in a high throughput format using automation, meaning thousands of compound-dose conditions can be tested in one experiment. We achieved this using pooled primary human hepatocytes, which are often fragile and expensive. By systemizing our automation and quality control processes, we were able to run over 120+ batches on the same donor pool with incredible reproducibility and consistency. We did this while integrating many readouts per well, whereas many existing toxicity assays only do a single readout. Our multiplexed approach provides far more data per experiment enabling us to measure 10-20 different toxicity phenotypes such as apoptosis, necrosis, mitochondrial fission, endoplasmic reticulum stress, stress granule formation, microtubules, and more all from a single well on a 384-well plate! The combination of scale, high content information, and data quality is exactly what is needed to train highly accurate AI models in biology. If you're interested, please explore the dataset in the comments below and let me know if you want to chat about the details!
0:19

To replace animal testing with AI, we need MASSIVE human datasets. Today, we're thrilled to share Axiom's new data exploration tool, providing the ability to visually explore the world's largest primary human liver toxicity dataset. Built with Axiom's proprietary wetlab protocols, our dataset includes detailed liver toxicity profiles for over 100,000 distinct molecules. The key to this dataset is our ability to do high-throughput, multiplexed high-content screening with primary human liver cells. Traditionally, toxicity assays either sacrifice throughput or sacrifice biological relevance (using easy-to-grow immortalized cell lines instead of real human cells). We managed to combine throughput, physiological relevance, and multiplexing in one platform. The assays run in a high throughput format using automation, meaning thousands of compound-dose conditions can be tested in one experiment. We achieved this using pooled primary human hepatocytes, which are often fragile and expensive. By systemizing our automation and quality control processes, we were able to run over 120+ batches on the same donor pool with incredible reproducibility and consistency. We did this while integrating many readouts per well, whereas many existing toxicity assays only do a single readout. Our multiplexed approach provides far more data per experiment enabling us to measure 10-20 different toxicity phenotypes such as apoptosis, necrosis, mitochondrial fission, endoplasmic reticulum stress, stress granule formation, microtubules, and more all from a single well on a 384-well plate! The combination of scale, high content information, and data quality is exactly what is needed to train highly accurate AI models in biology. If you're interested, please explore the dataset in the comments below and let me know if you want to chat about the details!

Brandon White

25,117 görüntüleme • 1 yıl önce

We’ve seen humanoid robots walk around for a while, but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:
0:18

We’ve seen humanoid robots walk around for a while, but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:

Yanjie Ze

75,515 görüntüleme • 2 yıl önce

Figure is aiming to develop the world’s largest and most diverse real-world humanoid pretraining dataset. For this purpose, they’re partnering with Brookfield, a global asset manager overseeing $1 trillion in assets, including 100,000 residential units, 500M square feet of commercial office space, and 160M square feet of logistics space. The data collected from this collaboration will be used to train Figure’s Helix AI model, enabling humanoids to perform tasks autonomously in real-world environments designed for humans. In addition to data collection, the partnership will explore support for next-generation GPU data centers, real estate for robotic training environments, and commercial use cases across Brookfield’s global footprint.
0:17

Figure is aiming to develop the world’s largest and most diverse real-world humanoid pretraining dataset. For this purpose, they’re partnering with Brookfield, a global asset manager overseeing $1 trillion in assets, including 100,000 residential units, 500M square feet of commercial office space, and 160M square feet of logistics space. The data collected from this collaboration will be used to train Figure’s Helix AI model, enabling humanoids to perform tasks autonomously in real-world environments designed for humans. In addition to data collection, the partnership will explore support for next-generation GPU data centers, real estate for robotic training environments, and commercial use cases across Brookfield’s global footprint.

The Humanoid Hub

88,600 görüntüleme • 1 yıl önce

🚀 Introducing EgoExo Forge - built on top of Rerun, Gradio, and Hugging Face hub (I’ll be in San Francisco July 21–29 — if you’re into robotics, egocentric AI, large-scale data collection, or just want to chat, DM me!) In my opinion, large-scale, diverse, and high-quality data is still the largest bottleneck for generalized robotics deployment. I believe that some version of imitation learning from human examples will be the most scalable + clean way to train humanoid robots 🤖 (similar to what Tesla did for Full Self Driving). Teleop is too expensive to collect a large enough dataset in a reasonable manner, so passive collection via egocentric (and in certain cases, exocentric) views feels like the right bet. Over the past few months, I've been trying to build out the scaffolding for this and using Rerun as my underlying infrastructure. Data being collected needs to be easily inspectable + time series and rerun provides the right tooling for this. My goal is to first build out a ground truth representative dataset from already existing open source data, generate some reasonable baselines, and then go out and collect my own data that adheres to the defined schema. 🔍 Starting with open-source datasets 1. EgoDex from Apple 2. HOCap from Nvidia and the University of Texas at Dallas 3. Assembly101 from Meta All these different datasets have different sensor configurations + annotations, so my goal with egoexo-forge is to have one consistent labeling scheme + data layout. I built a data pipeline that aligns all of the different datasets in one general schema assuming the COCO133 keypoint layout that allows for exo+ego, ego only, or exo only Since the scaffolding is already there, it becomes MUCH easier to add other datasets. So the next ones that I'll be including are HD-EPIC kitchens dataset, HOT3D, and finally my own personal iPhone + insta360 go collection method. Once I have a diverse variety of datasets, I'll double down on what I believe to be the key algorithms required to make useful data for imitation learning 📊 1. Camera Pose estimation via SLAM/SFM for ego perspective (and automatic calibration for exo) 2. Human pose estimation for both egocentric + exocentric views 3. Metric 3D reconstruction + object tracking I'll be setting up reasonable open-source baselines for each of these to validate that these datasets work, and then finally try to use the generated datasets for some imitation learning via the pi0-lerobot repo I've been working on. I plan on making a blog post + providing more info on all of this in the near future so stay tuned
0:18

🚀 Introducing EgoExo Forge - built on top of Rerun, Gradio, and Hugging Face hub (I’ll be in San Francisco July 21–29 — if you’re into robotics, egocentric AI, large-scale data collection, or just want to chat, DM me!) In my opinion, large-scale, diverse, and high-quality data is still the largest bottleneck for generalized robotics deployment. I believe that some version of imitation learning from human examples will be the most scalable + clean way to train humanoid robots 🤖 (similar to what Tesla did for Full Self Driving). Teleop is too expensive to collect a large enough dataset in a reasonable manner, so passive collection via egocentric (and in certain cases, exocentric) views feels like the right bet. Over the past few months, I've been trying to build out the scaffolding for this and using Rerun as my underlying infrastructure. Data being collected needs to be easily inspectable + time series and rerun provides the right tooling for this. My goal is to first build out a ground truth representative dataset from already existing open source data, generate some reasonable baselines, and then go out and collect my own data that adheres to the defined schema. 🔍 Starting with open-source datasets 1. EgoDex from Apple 2. HOCap from Nvidia and the University of Texas at Dallas 3. Assembly101 from Meta All these different datasets have different sensor configurations + annotations, so my goal with egoexo-forge is to have one consistent labeling scheme + data layout. I built a data pipeline that aligns all of the different datasets in one general schema assuming the COCO133 keypoint layout that allows for exo+ego, ego only, or exo only Since the scaffolding is already there, it becomes MUCH easier to add other datasets. So the next ones that I'll be including are HD-EPIC kitchens dataset, HOT3D, and finally my own personal iPhone + insta360 go collection method. Once I have a diverse variety of datasets, I'll double down on what I believe to be the key algorithms required to make useful data for imitation learning 📊 1. Camera Pose estimation via SLAM/SFM for ego perspective (and automatic calibration for exo) 2. Human pose estimation for both egocentric + exocentric views 3. Metric 3D reconstruction + object tracking I'll be setting up reasonable open-source baselines for each of these to validate that these datasets work, and then finally try to use the generated datasets for some imitation learning via the pi0-lerobot repo I've been working on. I plan on making a blog post + providing more info on all of this in the near future so stay tuned

Pablo Vela

37,592 görüntüleme • 1 yıl önce

(1/6) X-Humanoid 🤖: Scaling up data for Humanoid Robots. We convert human daily activity videos (from Ego-Exo4D) into humanoid videos (i.e., Tesla Optimus) performing tasks like cooking or fixing a bike. This data can be potentially used to train robot policies and world models. 🔥 Project page: Paper link:
0:11

(1/6) X-Humanoid 🤖: Scaling up data for Humanoid Robots. We convert human daily activity videos (from Ego-Exo4D) into humanoid videos (i.e., Tesla Optimus) performing tasks like cooking or fixing a bike. This data can be potentially used to train robot policies and world models. 🔥 Project page: Paper link:

Mike Shou

88,752 görüntüleme • 10 ay önce