Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

🤷What if we want to learn from human data... without human data? In our work NIL (No-data Imitation Learning) #CVPR2026, we explore a simple but ambitious question: Can robots learn directly from AI-generated videos without any curated demonstration data? 🔗

breadli428's profile picture

Chenhao Li

7,482 subscribers

22,062 Aufrufe • vor 2 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

The most frustrating part of imitation learning is collecting huge amounts of teleop data. But why teleop robots when robots can learn by watching us? Introducing Point Policy, a novel framework that enables robots to learn from human videos without any teleop, sim2real, or RL.
0:40

The most frustrating part of imitation learning is collecting huge amounts of teleop data. But why teleop robots when robots can learn by watching us? Introducing Point Policy, a novel framework that enables robots to learn from human videos without any teleop, sim2real, or RL.

Siddhant Haldar

69,056 Aufrufe • vor 1 Jahr

YouTube is a LARGE dataset of demonstration videos to train Generalist robot agents, but lacks action data. How can we learn DEXTEROUS skills from them? In #CoRL2024, we explore the problem of learning a Generalist Piano Playing agent from YouTube videos.
0:29

YouTube is a LARGE dataset of demonstration videos to train Generalist robot agents, but lacks action data. How can we learn DEXTEROUS skills from them? In #CoRL2024, we explore the problem of learning a Generalist Piano Playing agent from YouTube videos.

Julen Urain

42,451 Aufrufe • vor 1 Jahr

Imitation learning is great, but needs us to have (near) optimal data. We throw away most other data (failures, evaluation data, suboptimal data, undirected play data), even though this data can be really useful and way cheaper! In our new work - RISE, we show a simple way to *use all of this non-optimal data to robustify imitation learning* with minimal requirements beyond BC. Key idea: use non-expert data to learn how to *recover* back to expert data with a minimal frills offline RL that works under sparse data coverage. Allows usage of *all* available data, not just expert data - never throw your data away! Paper: Website: A 🧵(1/10)
0:57

Imitation learning is great, but needs us to have (near) optimal data. We throw away most other data (failures, evaluation data, suboptimal data, undirected play data), even though this data can be really useful and way cheaper! In our new work - RISE, we show a simple way to *use all of this non-optimal data to robustify imitation learning* with minimal requirements beyond BC. Key idea: use non-expert data to learn how to *recover* back to expert data with a minimal frills offline RL that works under sparse data coverage. Allows usage of *all* available data, not just expert data - never throw your data away! Paper: Website: A 🧵(1/10)

Abhishek Gupta

20,619 Aufrufe • vor 9 Monaten

Archetype AI published a new paper demonstrating how an AI model can encode and predict physical behaviours. Newton, the startup's ‘Large Behavior Model’, can learn complex physics principles directly from raw sensor data, without any human guidance.
1:07

Archetype AI published a new paper demonstrating how an AI model can encode and predict physical behaviours. Newton, the startup's ‘Large Behavior Model’, can learn complex physics principles directly from raw sensor data, without any human guidance.

Brett Adcock

25,962 Aufrufe • vor 1 Jahr

We discovered an emergent property of VLAs like π0/π0.5/π0.6: as we scale up pre-training, the model learns to align human videos and robot data! This gives us a simple way to leverage human videos. Once π0.5 knows how to control robots, it can naturally learn from human video.
2:02

We discovered an emergent property of VLAs like π0/π0.5/π0.6: as we scale up pre-training, the model learns to align human videos and robot data! This gives us a simple way to leverage human videos. Once π0.5 knows how to control robots, it can naturally learn from human video.

Physical Intelligence

1,184,714 Aufrufe • vor 8 Monaten

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵
0:55

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵

Mahi Shafiullah 🏠🤖

94,652 Aufrufe • vor 1 Monat

1/ 🧠Humans are the best robot data source! 2/ 👓Human egocentric video is rich in quantity, but poor in quality. 3/ Beyond scaling data, smarter representation and architecture matter just as much. 4/ Want an open-source framework to train your own learn-from-human-data robot policy? 🚀We introduce HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Robot-Data-Free ✦ Just 30 min of data per task ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Easy-to-Implement Let's squeeze every bit of signal out of human data! 🌐 Website: 📄 Paper: 💻 Code: 📹 Video: 🧵 1/n
1:28

1/ 🧠Humans are the best robot data source! 2/ 👓Human egocentric video is rich in quantity, but poor in quality. 3/ Beyond scaling data, smarter representation and architecture matter just as much. 4/ Want an open-source framework to train your own learn-from-human-data robot policy? 🚀We introduce HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Robot-Data-Free ✦ Just 30 min of data per task ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Easy-to-Implement Let's squeeze every bit of signal out of human data! 🌐 Website: 📄 Paper: 💻 Code: 📹 Video: 🧵 1/n

Zhi (Leo) Wang

110,230 Aufrufe • vor 2 Monaten

Over the last few months, we’ve been thinking about how to learn from “off-domain” data - data from non-robot sources like video or simulation. These data sources are not quite good enough to learn policies (even monolithic VLA models) directly, but they still contain lots of information that can be useful for generalizable robot control. How can we develop robot learning models that are able to make use of this type of data for generalizable control? In new work, that we call HAMSTER, we show that VLMs can be useful for enabling robotic learning from off-domain data, but specifically when used through hierarchical VLA architectures. We show that this class of models can learn generalizable robot policies for the real world from large-scale, off-domain data. A 🧵 (1/10)
1:00

Over the last few months, we’ve been thinking about how to learn from “off-domain” data - data from non-robot sources like video or simulation. These data sources are not quite good enough to learn policies (even monolithic VLA models) directly, but they still contain lots of information that can be useful for generalizable robot control. How can we develop robot learning models that are able to make use of this type of data for generalizable control? In new work, that we call HAMSTER, we show that VLMs can be useful for enabling robotic learning from off-domain data, but specifically when used through hierarchical VLA architectures. We show that this class of models can learn generalizable robot policies for the real world from large-scale, off-domain data. A 🧵 (1/10)

Abhishek Gupta

11,994 Aufrufe • vor 1 Jahr

Helix is now learning directly from human video data We have already trained on data collected in the real world, including Brookfield residential units To our knowledge, this is the first instance of a humanoid robot learning navigation end-to-end using only human video
0:24

Helix is now learning directly from human video data We have already trained on data collected in the real world, including Brookfield residential units To our knowledge, this is the first instance of a humanoid robot learning navigation end-to-end using only human video

Figure

46,041 Aufrufe • vor 11 Monaten

“we’ll have to have some sort of Universal Basic Income (UBI)” Elon Musk But who is going to pay? We believe our data! HUDI = Human Data Income We are building a Universal Basic Income based on our data. Our data is our asset, the new oil, but we have no transparency nor profit from it. Help us build the Human Data Income: we made great progress, but it’s just the beginning!
0:32

“we’ll have to have some sort of Universal Basic Income (UBI)” Elon Musk But who is going to pay? We believe our data! HUDI = Human Data Income We are building a Universal Basic Income based on our data. Our data is our asset, the new oil, but we have no transparency nor profit from it. Help us build the Human Data Income: we made great progress, but it’s just the beginning!

HUDI

11,400,385 Aufrufe • vor 2 Jahren

Presenting DemoDiffusion: An extremely simple approach enabling a pre-trained 'generalist' diffusion policy to follow a human-demonstration for a novel task during inference One-shot human imitation *without* requiring any paired human-robot data or online RL 🙂 1/n
0:30

Presenting DemoDiffusion: An extremely simple approach enabling a pre-trained 'generalist' diffusion policy to follow a human-demonstration for a novel task during inference One-shot human imitation *without* requiring any paired human-robot data or online RL 🙂 1/n

Homanga Bharadhwaj

32,919 Aufrufe • vor 1 Jahr

🤖WE HUMANS ARE TRAINING OUR AI REPLACEMENTS Humanoid robots are learning by watching humans work. Workers wear devices that capture egocentric data, showing robots exactly how tasks look from a human perspective. We are not just building robots. We are handing them our job's playbook.
0:29

🤖WE HUMANS ARE TRAINING OUR AI REPLACEMENTS Humanoid robots are learning by watching humans work. Workers wear devices that capture egocentric data, showing robots exactly how tasks look from a human perspective. We are not just building robots. We are handing them our job's playbook.

Coin Bureau

20,180 Aufrufe • vor 2 Monaten

Can we synthesize 3D human-scene interactions without learning from any 3D data? Yes! Check out Lei Li's GenZI, a novel zero-shot approach to generating 3D interactions by distilling priors from large vision-language models.
0:22

Can we synthesize 3D human-scene interactions without learning from any 3D data? Yes! Check out Lei Li's GenZI, a novel zero-shot approach to generating 3D interactions by distilling priors from large vision-language models.

Angela Dai

106,939 Aufrufe • vor 2 Jahren

Heard some people like wheels？😁 Humanoid robots are the ideal form of general-purpose robots (perfect for general AI and human-derived data). They can work without wheels — but they can also have wheels if they want. Whatever works.
1:14

Heard some people like wheels？😁 Humanoid robots are the ideal form of general-purpose robots (perfect for general AI and human-derived data). They can work without wheels — but they can also have wheels if they want. Whatever works.

Unitree

8,164,357 Aufrufe • vor 3 Monaten

Imagine robots learning new skills—without any robot data. Today, we're excited to release EgoZero: our first steps in training robot policies that operate in unseen environments, solely from data collected through humans wearing Aria smart glasses. 🧵👇
1:24

Imagine robots learning new skills—without any robot data. Today, we're excited to release EgoZero: our first steps in training robot policies that operate in unseen environments, solely from data collected through humans wearing Aria smart glasses. 🧵👇

Lerrel Pinto

42,576 Aufrufe • vor 1 Jahr

Model progress is no longer constrained by architecture, but by access to high-quality, human-generated data. Scraped internet data is finite, low-signal and increasingly synthetic. Kled AI pays real people to upload authentic, real world content task by task in their mobile app. We are super excited to back Avi Patel and Kled AI as they build a massive data marketplace. If you are in need of a great data partner, reach out! More from us here (we were not investors when we filmed):
23:44

Model progress is no longer constrained by architecture, but by access to high-quality, human-generated data. Scraped internet data is finite, low-signal and increasingly synthetic. Kled AI pays real people to upload authentic, real world content task by task in their mobile app. We are super excited to back Avi Patel and Kled AI as they build a massive data marketplace. If you are in need of a great data partner, reach out! More from us here (we were not investors when we filmed):

Nichole Wischoff

36,974 Aufrufe • vor 4 Monaten

ChatGPT connecting to external data sources offers a glimpse into what AI interoperability looks like. Ask any question in ChatGPT and it can now pull data from Box. The future of AI is AI Agents that can access your systems and data to complete any task for you.
0:43

ChatGPT connecting to external data sources offers a glimpse into what AI interoperability looks like. Ask any question in ChatGPT and it can now pull data from Box. The future of AI is AI Agents that can access your systems and data to complete any task for you.

Aaron Levie

57,745 Aufrufe • vor 1 Jahr

Train the AI behind real-world robots We’re hiring Data Labelers to annotate images and videos to train Optimus & autonomous systems. You’ll work with real production data & directly impact how these systems learn and operate in the real world Come join!
0:33

Train the AI behind real-world robots We’re hiring Data Labelers to annotate images and videos to train Optimus & autonomous systems. You’ll work with real production data & directly impact how these systems learn and operate in the real world Come join!

Tesla Recruiting

661,884 Aufrufe • vor 3 Monaten

Imitation learning works™ – but you need good data 🥹 How to get high-quality visuotactile demos from a bimanual robot with multifingered hands, and learn smooth policies? Check our new work “Learning Visuotactile Skills with Two Multifingered Hands”! 🙌
0:29

Imitation learning works™ – but you need good data 🥹 How to get high-quality visuotactile demos from a bimanual robot with multifingered hands, and learn smooth policies? Check our new work “Learning Visuotactile Skills with Two Multifingered Hands”! 🙌

Toru

61,584 Aufrufe • vor 2 Jahren

Eric Schmidt says war is no longer constrained by humans. It's AI vs AI. “No human can plan a battle… without reinforcement learning.” The real conflict lives in data centers and drone swarms. No human can follow it in real time.
1:17

Eric Schmidt says war is no longer constrained by humans. It's AI vs AI. “No human can plan a battle… without reinforcement learning.” The real conflict lives in data centers and drone swarms. No human can follow it in real time.

vitrupo

112,833 Aufrufe • vor 1 Jahr