Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

We (Pantheon) have been working on a robotics data quality pipeline that uncovered a series of major problems in public robotics datasets, especially for world modeling. To improve the quality of data available to open-source robotics, we're publishing annotations for four of the most popular datasets. Some examples of... show more

calix

12,273 subscribers

83,197 görüntüleme • 8 saat önce •via X (Twitter)

Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

37 Yorum

calix profil fotoğrafı

calix8 saat önce

1. Camera disconnection One of the wrist cameras in this episode froze, then disconnected. This would show up as pure noise to a world model.

calix profil fotoğrafı

calix8 saat önce

2. Missing gripper actions Despite the left gripper opening and closing, no actions are recorded. The world model is penalized for failing to predict the gripper's actions.

calix profil fotoğrafı

calix8 saat önce

3. Spiky actions In some datasets, action data contains sharp spikes. These are hard for a world action model with broad smoothness priors to fit.

calix profil fotoğrafı

calix8 saat önce

4. Shifted actions Action data is misaligned by a few frames for the grasp, but not the movement. This tricks the world model into predicting state changes too early or late.

calix profil fotoğrafı

calix8 saat önce

Read our full report: Credits to @sambhavgupta159 for uncovering and publishing this analysis. We're hiring for robotics foundation model research, software, data, and hardware engineering. We're also open to data collaborations.

"Leigh Marie" Braswell profil fotoğrafı

"Leigh Marie" Braswell7 saat önce

@PantheonInc Such important work! Data is king

Calvin Chen profil fotoğrafı

Calvin Chen8 saat önce

@PantheonInc bullish for looking at the data you’d be surprised how many don’t

Jannik Schilling profil fotoğrafı

Jannik Schilling7 saat önce

@PantheonInc Crazy how many issues we found in just two weeks...

carlo agostinelli profil fotoğrafı

carlo agostinelli8 saat önce

@PantheonInc I’ve been waiting for this

Aryan Shah profil fotoğrafı

Aryan Shah7 saat önce

@PantheonInc Dream team!!

Nikhil Suresh profil fotoğrafı

Nikhil Suresh7 saat önce

@hamostaf04 @PantheonInc Great work on the data!

Amir Fischer profil fotoğrafı

Amir Fischer7 saat önce

@PantheonInc @Jannikschg @arminsommer_ the european robotics mafia!

Kushal Thaman profil fotoğrafı

Kushal Thaman7 saat önce

@PantheonInc bot see good data, bot happy :)

Yang Fan Yun profil fotoğrafı

Yang Fan Yun7 saat önce

@PantheonInc let’s goooooo @calixo888 @nettedjay !!!!!! 🚀

Keval Shah profil fotoğrafı

Keval Shah7 saat önce

@PantheonInc Interesting read! Congrats to the team @_varunnair great to see you in action :)

arminsommer profil fotoğrafı

arminsommer7 saat önce

@PantheonInc @sambhavgupta159 is a beast! very nice work!

devansh profil fotoğrafı

devansh7 saat önce

@PantheonInc cool!!

Sonith profil fotoğrafı

Sonith7 saat önce

@PantheonInc 🙏

nathan chen profil fotoğrafı

nathan chen7 saat önce

@PantheonInc amazing read! cong to the team :D @Jannikschg

Sarah Su profil fotoğrafı

Sarah Su7 saat önce

@PantheonInc this is super cool!

Henry Yin profil fotoğrafı

Henry Yin7 saat önce

@PantheonInc KALIX!!

Amulya Harish profil fotoğrafı

Amulya Harish7 saat önce

@PantheonInc The beginning

Victor Li profil fotoğrafı

Victor Li7 saat önce

@PantheonInc 🔥🔥

Henry Lien profil fotoğrafı

Henry Lien7 saat önce

@PantheonInc 🚀🚀🚀

npjd profil fotoğrafı

npjd7 saat önce

@PantheonInc so amazing calix

Osmond profil fotoğrafı

Osmond7 saat önce

@PantheonInc quietly cooking!!

OnchainLoving.ETH profil fotoğrafı

OnchainLoving.ETH7 saat önce

@PantheonInc finally some real data rigor

Varun Nair profil fotoğrafı

Varun Nair7 saat önce

@PantheonInc Data Interpretability is huge. These are fixable issues and the human data community should know the aspects that must be improved. Overall great for the community as a whole!

Benedict Wolff profil fotoğrafı

Benedict Wolff7 saat önce

@PantheonInc Crushing it!

Eric Li profil fotoğrafı

Eric Li7 saat önce

@PantheonInc Well done Mr Gupta @sambhavgupta159

hamza mostafa profil fotoğrafı

hamza mostafa7 saat önce

@PantheonInc very cool work!!

Kole Lee profil fotoğrafı

Kole Lee7 saat önce

@PantheonInc letsgoo

Daniel Ceballos profil fotoğrafı

Daniel Ceballos7 saat önce

@PantheonInc Awesome results!!!

Zain Javaid profil fotoğrafı

Zain Javaid7 saat önce

@PantheonInc awesome work! congrats pantheon

joseph profil fotoğrafı

joseph7 saat önce

@PantheonInc long time coming 💪

Alexander Yevchenko profil fotoğrafı

Alexander Yevchenko7 saat önce

@PantheonInc thank you!!

Zuhair Lakhani profil fotoğrafı

Zuhair Lakhani7 saat önce

@PantheonInc Thank you for looking at the data

Benzer Videolar

Today we're launching Hebbian Robotics (YC S26) Hebbian Robotics (YC S26) . We build APIs for evaluating data quality, without training a robotics model. Data vendors use our API to search and analyze Physical AI data at scale, getting on-demand quality signals and metrics without managing infrastructure. High-quality data is the bottleneck to the next frontier of intelligence in robotics. Buyers want data they can trust. Data providers are focused on operational excellence. They don't have the bandwidth to understand the quality of their data at scale. We're fixing this. Vendors use our APIs to understand the impact of their decisions on their data quality: - Buyers demanding per-collector hour limits? Prove compliance while we stay ahead of the workarounds. - Changed your collection SOPs? Monitor for quality drift and regression. - Suspect duplicate samples? Verify with a single API call. Before Hebbian, I trained AI models for industrial cleaning robots, and Kingston Kuan spent years building large-scale infrastructure at Jane Street.

Today we're launching Hebbian Robotics (YC S26) Hebbian Robotics (YC S26) . We build APIs for evaluating data quality, without training a robotics model. Data vendors use our API to search and analyze Physical AI data at scale, getting on-demand quality signals and metrics without managing infrastructure. High-quality data is the bottleneck to the next frontier of intelligence in robotics. Buyers want data they can trust. Data providers are focused on operational excellence. They don't have the bandwidth to understand the quality of their data at scale. We're fixing this. Vendors use our APIs to understand the impact of their decisions on their data quality: - Buyers demanding per-collector hour limits? Prove compliance while we stay ahead of the workarounds. - Changed your collection SOPs? Monitor for quality drift and regression. - Suspect duplicate samples? Verify with a single API call. Before Hebbian, I trained AI models for industrial cleaning robots, and Kingston Kuan spent years building large-scale infrastructure at Jane Street.

Brandon Ong

166,623 görüntüleme • 1 ay önce

1/ The data parsing market for robotics is just emerging, and it’s already valued in the tens of billions. Data parsing to build data models and software for Robotics is a $20B market in 2024 and is expected to reach $150B+ by 2034 Cherry AI Robotics has built a data parsing tool for robotics, which goes live next week, and will effectively make this project one of the pioneers in data parsing for robotics.

1/ The data parsing market for robotics is just emerging, and it’s already valued in the tens of billions. Data parsing to build data models and software for Robotics is a $20B market in 2024 and is expected to reach $150B+ by 2034 Cherry AI Robotics has built a data parsing tool for robotics, which goes live next week, and will effectively make this project one of the pioneers in data parsing for robotics.

CherryAI Robotics

90,786 görüntüleme • 1 yıl önce

Most capable generalist robotics models today are closed or at best, open weights. But robotics won’t reach its ChatGPT moment without real openness. That GPT moment was built on years of open tools and datasets such as Python, PyTorch, ImageNet and more, that let researchers inspect, reproduce, and build. Today, we’re introducing MolmoAct 2: a fully open-source action reasoning model for real-world robotics. We rethought and reshaped everything! 🧵👇

Most capable generalist robotics models today are closed or at best, open weights. But robotics won’t reach its ChatGPT moment without real openness. That GPT moment was built on years of open tools and datasets such as Python, PyTorch, ImageNet and more, that let researchers inspect, reproduce, and build. Today, we’re introducing MolmoAct 2: a fully open-source action reasoning model for real-world robotics. We rethought and reshaped everything! 🧵👇

Jiafei Duan

117,636 görüntüleme • 4 ay önce

Building valuable datasets takes dedication, skill and expertise. From robotics data to trend forecasting, we incubate cutting-edge datasets that generate lasting real-world value. Vana connects passionate entrepreneurs with our network of top backers, experts and data buyers. Thank you to Hub71, ecosystem mentors, and partners for joining us at the Vana Founders' Intensive in Abu Dhabi. Real builders. Real business. That's the Vana way.

Building valuable datasets takes dedication, skill and expertise. From robotics data to trend forecasting, we incubate cutting-edge datasets that generate lasting real-world value. Vana connects passionate entrepreneurs with our network of top backers, experts and data buyers. Thank you to Hub71, ecosystem mentors, and partners for joining us at the Vana Founders' Intensive in Abu Dhabi. Real builders. Real business. That's the Vana way.

vana

15,816 görüntüleme • 1 yıl önce

World Labs CEO Dr. Fei-Fei Li & SceniX Co-Founder Yunzhu Li on the data bottleneck in robotics and how world models help: "The lack of data in training, the lack of data in evaluation, this is very, very different from language models, where data is abundant on the internet." "And we know that in order for robotics to work, we have to somehow unlock the power of scaling law. But where does that come from?" "It's a profound problem that everybody's battling with in robotics." "We see a lot of unlock in being able to do this whole process through the modeling of the environments." "Being able to create these digital worlds... that's just going to unlock so much more potential for being able to replace all the costly and the unsafe data in the real environments with the data generated from the worlds for robots to be able to do scalable learning and evaluations." Fei-Fei Li Yunzhu Li martin_casado

World Labs CEO Dr. Fei-Fei Li & SceniX Co-Founder Yunzhu Li on the data bottleneck in robotics and how world models help: "The lack of data in training, the lack of data in evaluation, this is very, very different from language models, where data is abundant on the internet." "And we know that in order for robotics to work, we have to somehow unlock the power of scaling law. But where does that come from?" "It's a profound problem that everybody's battling with in robotics." "We see a lot of unlock in being able to do this whole process through the modeling of the environments." "Being able to create these digital worlds... that's just going to unlock so much more potential for being able to replace all the costly and the unsafe data in the real environments with the data generated from the worlds for robots to be able to do scalable learning and evaluations." Fei-Fei Li Yunzhu Li martin_casado

a16z

77,065 görüntüleme • 1 ay önce

New meta on the horizon - AI x Robotics?👀 "The ChatGPT moment for general robotics is just around the corner..." – Jensen Huang, CEO of NVIDIA Most profitable wallet on 7D leaderboard by Nansen 🧭 loaded a few Robotic plays: 🔷 $SAM - Crowdsourcing real-world datasets with robotic gaming 🔷 $HIBER - robust AI Infrastructure 🔷 $BREW - Intersection of robotics × AI × crypto 🔷 $MROBOTICS - mobile gaming and robotics company Another quality project to research: $XRT What else did I miss?🤔

New meta on the horizon - AI x Robotics?👀 "The ChatGPT moment for general robotics is just around the corner..." – Jensen Huang, CEO of NVIDIA Most profitable wallet on 7D leaderboard by Nansen 🧭 loaded a few Robotic plays: 🔷 $SAM - Crowdsourcing real-world datasets with robotic gaming 🔷 $HIBER - robust AI Infrastructure 🔷 $BREW - Intersection of robotics × AI × crypto 🔷 $MROBOTICS - mobile gaming and robotics company Another quality project to research: $XRT What else did I miss?🤔

Crypto Gideon(🧠,🧠)

15,027 görüntüleme • 1 yıl önce

Very excited to share what our research team at Covariant has been working on: RFM-1, our latest Robotics Foundation Model. Built on top of high-quality multimodal data the Covariant robotics fleet has collected over the years, this project embodies our commitment to pushing the boundaries of how AI can interact with and understand the physical world. We're eager to share this milestone with the community and invite feedback and collaboration. Read more at

Very excited to share what our research team at Covariant has been working on: RFM-1, our latest Robotics Foundation Model. Built on top of high-quality multimodal data the Covariant robotics fleet has collected over the years, this project embodies our commitment to pushing the boundaries of how AI can interact with and understand the physical world. We're eager to share this milestone with the community and invite feedback and collaboration. Read more at

Rocky Duan

31,887 görüntüleme • 2 yıl önce

Co-training has become a key part of the recipe for training large robotics models; it means that you mix some proportion of real robot data with other data sources, like simulation or egocentric human video data. This is especially important because robotics data tends to lack diversity which can be somewhat compensated for by the inclusion of these other modalities. And yet there has not been a sizable study on what constitute good practices for cotraining until now! We talk to Fanqi Lin and Jose Barreiros about their new work, a massive study which evaluated 89 policies over thousands of rollouts to tell us which forms of co-training were most useful for robotics. Watch episode 70 of RoboPapers, with Michael Cho and Chris Paxton, now!

Co-training has become a key part of the recipe for training large robotics models; it means that you mix some proportion of real robot data with other data sources, like simulation or egocentric human video data. This is especially important because robotics data tends to lack diversity which can be somewhat compensated for by the inclusion of these other modalities. And yet there has not been a sizable study on what constitute good practices for cotraining until now! We talk to Fanqi Lin and Jose Barreiros about their new work, a massive study which evaluated 89 policies over thousands of rollouts to tell us which forms of co-training were most useful for robotics. Watch episode 70 of RoboPapers, with Michael Cho and Chris Paxton, now!

RoboPapers

19,357 görüntüleme • 5 ay önce

Robotics in the age of FMs, cloud and Starlink. We've been rethinking robot AI — cloud-first, LLM-first approach. Fast to deploy. Modular. Data-centric. Built for long-horizon reasoning. Paper: Blog: 🧵 #Robotics #AI #LLM #VLM

Robotics in the age of FMs, cloud and Starlink. We've been rethinking robot AI — cloud-first, LLM-first approach. Fast to deploy. Modular. Data-centric. Built for long-horizon reasoning. Paper: Blog: 🧵 #Robotics #AI #LLM #VLM

Ashish Kapoor

10,185 görüntüleme • 11 ay önce

The Open Robotics AI Forum came to life in Seoul, thanks to the energy of our contributors and participants. Together we’re shaping an open future for robotics.

The Open Robotics AI Forum came to life in Seoul, thanks to the energy of our contributors and participants. Together we’re shaping an open future for robotics.

OpenMind

23,633 görüntüleme • 11 ay önce

A few months ago we set out to understand exactly how our models can be used to accelerate progress in robotics and other AI systems that don’t just represent the world, but are actually able to interact with it. We’ve been collaborating with leading companies in the space and we’ve developed an initial approach that we're now opening up to accelerate development across the industry. Our fifth announcement is GWM Robotics. GWM Robotics is a learned simulator that generates synthetic data for scalable robot training and policy evaluation, removing the bottlenecks of physical hardware.

A few months ago we set out to understand exactly how our models can be used to accelerate progress in robotics and other AI systems that don’t just represent the world, but are actually able to interact with it. We’ve been collaborating with leading companies in the space and we’ve developed an initial approach that we're now opening up to accelerate development across the industry. Our fifth announcement is GWM Robotics. GWM Robotics is a learned simulator that generates synthetic data for scalable robot training and policy evaluation, removing the bottlenecks of physical hardware.

Runway

35,040 görüntüleme • 9 ay önce

Sergey Levine (Sergey Levine) is one of the world's top robotics researchers and co-founder of Physical Intelligence. We talked about where humanoid robotics is today, thoughts on the Chinese robotics ecosystem, and his predictions for future timelines. In this episode: • Current state of robotics and surprising capabilities so far • Chinese robotics compared to US ecosystem • If OpenAI and Anthropic got into robotics • His top robotics research paper recommendation • Predictions for when humanoid robotics will land Where to watch: • YouTube - • Spotify - • Apple Podcasts - • Transcript - Thank you to the sponsor of this episode for supporting my work: • WorkOS: makes your app Enterprise Ready with easy to use APIs to add SSO, SCIM, RBAC, and more in just a few lines of code, check them out at Chapters: 00:00 Intro 00:37 Where are we today 04:20 Most surprising capabilities so far 07:03 The most inspiring real world robotics 08:36 If OpenAI or Anthropic got into robotics 10:22 Chinese robotics 13:15 Will one lab breakout from the rest 16:59 Thoughts on a concrete roadmap 21:03 Generalization and demonstrating it 26:04 Types of data and which is best for robotics 34:34 Why humanoid robotics differs from Waymo 37:10 If humanoid robotics failed here is why 39:55 Are there hot take modeling architectures in robotics 42:05 Thoughts on AI safety in robotics 46:44 Top robotics research paper recommendation 49:35 Why is Boston Dynamics less top of mind 53:47 Advice for his younger self 56:42 Outro

Sergey Levine (Sergey Levine) is one of the world's top robotics researchers and co-founder of Physical Intelligence. We talked about where humanoid robotics is today, thoughts on the Chinese robotics ecosystem, and his predictions for future timelines. In this episode: • Current state of robotics and surprising capabilities so far • Chinese robotics compared to US ecosystem • If OpenAI and Anthropic got into robotics • His top robotics research paper recommendation • Predictions for when humanoid robotics will land Where to watch: • YouTube - • Spotify - • Apple Podcasts - • Transcript - Thank you to the sponsor of this episode for supporting my work: • WorkOS: makes your app Enterprise Ready with easy to use APIs to add SSO, SCIM, RBAC, and more in just a few lines of code, check them out at Chapters: 00:00 Intro 00:37 Where are we today 04:20 Most surprising capabilities so far 07:03 The most inspiring real world robotics 08:36 If OpenAI or Anthropic got into robotics 10:22 Chinese robotics 13:15 Will one lab breakout from the rest 16:59 Thoughts on a concrete roadmap 21:03 Generalization and demonstrating it 26:04 Types of data and which is best for robotics 34:34 Why humanoid robotics differs from Waymo 37:10 If humanoid robotics failed here is why 39:55 Are there hot take modeling architectures in robotics 42:05 Thoughts on AI safety in robotics 46:44 Top robotics research paper recommendation 49:35 Why is Boston Dynamics less top of mind 53:47 Advice for his younger self 56:42 Outro

Ryan Peterman

70,752 görüntüleme • 1 ay önce

Unreal Engine 5 is becoming the go-to platform for robotics teams, not just as a simulator but as a synthetic data factory powering the next generation of robotics systems🦾 From synthetic data to sim-to-real transfer, read our piece on how UE5 is quietly reshaping how robots learn to see, move, and act in the real world:

Unreal Engine 5 is becoming the go-to platform for robotics teams, not just as a simulator but as a synthetic data factory powering the next generation of robotics systems🦾 From synthetic data to sim-to-real transfer, read our piece on how UE5 is quietly reshaping how robots learn to see, move, and act in the real world:

Unreal Engine

61,187 görüntüleme • 5 ay önce

The open-source AI community just got a new home for their data workflows. 🤗 Hugging Face is now available in Adaptive Data. Pull datasets directly into a platform that evolves with the problems you're solving.

The open-source AI community just got a new home for their data workflows. 🤗 Hugging Face is now available in Adaptive Data. Pull datasets directly into a platform that evolves with the problems you're solving.

adaption

55,194 görüntüleme • 5 ay önce

🤖 OrionX Robotics is actively engaged in a fundraising round to accelerate growth, scale operations, and advance the development of next-generation autonomous robotics systems. We look forward to partnering with investors who share our vision for the future of intelligent robotics.

🤖 OrionX Robotics is actively engaged in a fundraising round to accelerate growth, scale operations, and advance the development of next-generation autonomous robotics systems. We look forward to partnering with investors who share our vision for the future of intelligent robotics.

OrionX Robotics

12,345 görüntüleme • 3 ay önce

Figure just crossed 86,000 weekly active users uploading data We're building the largest and most diverse robotics dataset in the world The website has a live map viewer showing each pixel of data as it's uploaded around the world

Figure just crossed 86,000 weekly active users uploading data We're building the largest and most diverse robotics dataset in the world The website has a live map viewer showing each pixel of data as it's uploaded around the world

Brett Adcock

113,943 görüntüleme • 13 gün önce

NVIDIA CEO Jensen Huang: Elon Musk is in a phenomenal position to lead the future of AI, autonomous vehicles, and humanoid robotics. “Collecting world data is very expensive, and Elon has a great advantage because, number one, his AI factory for his cars is fantastic, has a lot of NVIDIA gear in it, and he has a very large fleet of cars on the road that allows him to collect a lot of data. So I think he has just a phenomenal position, and he's been working on this for a long time, and so he's going to be in a great position to take advantage of it. I know Elon's attitude towards AI, and he's very optimistic about its future, and obviously he's working on some of the most important AI areas. xAI is working on foundation, cognitive intelligence AI. Tesla's working on autonomous vehicles, and Optimus is for humanoid robotics. These three areas of AI are the three most important areas.”

NVIDIA CEO Jensen Huang: Elon Musk is in a phenomenal position to lead the future of AI, autonomous vehicles, and humanoid robotics. “Collecting world data is very expensive, and Elon has a great advantage because, number one, his AI factory for his cars is fantastic, has a lot of NVIDIA gear in it, and he has a very large fleet of cars on the road that allows him to collect a lot of data. So I think he has just a phenomenal position, and he's been working on this for a long time, and so he's going to be in a great position to take advantage of it. I know Elon's attitude towards AI, and he's very optimistic about its future, and obviously he's working on some of the most important AI areas. xAI is working on foundation, cognitive intelligence AI. Tesla's working on autonomous vehicles, and Optimus is for humanoid robotics. These three areas of AI are the three most important areas.”

DogeDesigner

142,845 görüntüleme • 1 ay önce

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

Chris Paxton

20,486 görüntüleme • 1 yıl önce