正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Training robots for the open world needs diverse data But collecting robot demos in the wild is hard! Presenting DexWild 🙌🏕️ Human data collection system that works in diverse environments, without robots 💪🦾 Human + Robot Cotraining pipeline that unlocks generalization 🧵👇

Tony Tao

1,054 subscribers

97,762 次观看 • 1 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

the micro1 robotics lab: real world data for intelligent models that co-exist in the physical world. we’re in-the-wild across 75 countries in 6,000+ unique environments collecting data. diverse movements, objects, and settings. the future of AI is as human as you can imagine. join us to start training robots today (link in comments).

the micro1 robotics lab: real world data for intelligent models that co-exist in the physical world. we’re in-the-wild across 75 countries in 6,000+ unique environments collecting data. diverse movements, objects, and settings. the future of AI is as human as you can imagine. join us to start training robots today (link in comments).

Ali Ansari

209,148 次观看 • 8 个月前

Can robots self-improve by collecting data autonomously🤖? Introducing SOAR: a system for large-scale autonomous data collection 🚀 and autonomous improvement📈of a multi-task language-conditioned policy in diverse scenes without human interventions .

Can robots self-improve by collecting data autonomously🤖? Introducing SOAR: a system for large-scale autonomous data collection 🚀 and autonomous improvement📈of a multi-task language-conditioned policy in diverse scenes without human interventions .

Paul Zhou

47,739 次观看 • 2 年前

Imagine robots learning new skills—without any robot data. Today, we're excited to release EgoZero: our first steps in training robot policies that operate in unseen environments, solely from data collected through humans wearing Aria smart glasses. 🧵👇

Imagine robots learning new skills—without any robot data. Today, we're excited to release EgoZero: our first steps in training robot policies that operate in unseen environments, solely from data collected through humans wearing Aria smart glasses. 🧵👇

Lerrel Pinto

42,673 次观看 • 1 年前

Human Data. Robot Data. Virtual Data. BrainCo Dexterous Data Matrix connects diverse data sources to establish a closed-loop data ecosystem, enabling robots to learn from real-world experiences and achieve more natural interactions. #BrainCo #EmbodiedAI #FutureRobotics

Human Data. Robot Data. Virtual Data. BrainCo Dexterous Data Matrix connects diverse data sources to establish a closed-loop data ecosystem, enabling robots to learn from real-world experiences and achieve more natural interactions. #BrainCo #EmbodiedAI #FutureRobotics

Tulsi Soni

67,586 次观看 • 2 个月前

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

42,864 次观看 • 9 个月前

For roboticists, one challenge towers above the others: there isn’t enough data. To accelerate the deployment of intelligent robots in the real world, MIT CSAIL’s "LucidSim" uses genAI & physics engines to create diverse & realistic virtual training grounds for robots. W/o any real-world data, their robot achieved expert-level performance in difficult tasks:

For roboticists, one challenge towers above the others: there isn’t enough data. To accelerate the deployment of intelligent robots in the real world, MIT CSAIL’s "LucidSim" uses genAI & physics engines to create diverse & realistic virtual training grounds for robots. W/o any real-world data, their robot achieved expert-level performance in difficult tasks:

MIT CSAIL

82,970 次观看 • 2 年前

This year is truly the dawn of the “robot data oil” rush--spawning all kinds of pick-and-shovel tools to dig it up. X Square Robot, which offers in-home humanoid cleaning services, has just launched a full UMI/Egocentric data collection system: QUANXTA Zero. Real robots delivering actual on-site services is one powerful data collection method. Meanwhile, bodiless (no physical robot) UMI/Ego-centric( human-centric)data pipelines are also surging in popularity. Well,everyone’s racing to distill the physical world and extract the data that truly matters for advancing robot intelligence.

This year is truly the dawn of the “robot data oil” rush--spawning all kinds of pick-and-shovel tools to dig it up. X Square Robot, which offers in-home humanoid cleaning services, has just launched a full UMI/Egocentric data collection system: QUANXTA Zero. Real robots delivering actual on-site services is one powerful data collection method. Meanwhile, bodiless (no physical robot) UMI/Ego-centric( human-centric)data pipelines are also surging in popularity. Well,everyone’s racing to distill the physical world and extract the data that truly matters for advancing robot intelligence.

CyberRobo

21,950 次观看 • 4 个月前

How can roboticists scale human manipulation data in the wild? EgoHumanoid builds a comprehensive pipeline which unlocks mobile manipulation training for robots — meaning that you can walk around and collect data for your robot anywhere you go. There are two key components to their recipe: view alignment, to reduce mismatch between human and robot camera views, and action alignment, to make sure human actions can be converted into robot actions. This makes it easier than ever to train robot policies and deploy them in previously-unseen environments. Modi Shi joins us to tell us the details. Watch Episode #96 of RoboPapers with Michael Cho ✈️ CoRL and Jiafei Duan✈️CoRL2026 today!

How can roboticists scale human manipulation data in the wild? EgoHumanoid builds a comprehensive pipeline which unlocks mobile manipulation training for robots — meaning that you can walk around and collect data for your robot anywhere you go. There are two key components to their recipe: view alignment, to reduce mismatch between human and robot camera views, and action alignment, to make sure human actions can be converted into robot actions. This makes it easier than ever to train robot policies and deploy them in previously-unseen environments. Modi Shi joins us to tell us the details. Watch Episode #96 of RoboPapers with Michael Cho ✈️ CoRL and Jiafei Duan✈️CoRL2026 today!

RoboPapers

14,513 次观看 • 3 个月前

HumanX: a scalable framework that converts single monocular human videos into agile, generalizable interaction skills for humanoid robots without task-specific rewards. Core parts: - XGen: retargets human motion + synthesizes diverse physically plausible training data (via physics-driven object trajectories and augmentation). - XMimic: unified imitation pipeline for robust generalization. Shows promising results for both Blind (no external sensors) and MoCap modes.

HumanX: a scalable framework that converts single monocular human videos into agile, generalizable interaction skills for humanoid robots without task-specific rewards. Core parts: - XGen: retargets human motion + synthesizes diverse physically plausible training data (via physics-driven object trajectories and augmentation). - XMimic: unified imitation pipeline for robust generalization. Shows promising results for both Blind (no external sensors) and MoCap modes.

The Humanoid Hub

20,358 次观看 • 9 个月前

LingBot-VLA 2.0 is an impressive new embodied model. Open source and is trained across diverse robot configurations, from single-arm robots to humanoid platforms. It packs 60K hours of curated robot and human data into one generalist policy. It improves robots on difficult long-horizon tasks. Great release by Robbyant.

LingBot-VLA 2.0 is an impressive new embodied model. Open source and is trained across diverse robot configurations, from single-arm robots to humanoid platforms. It packs 60K hours of curated robot and human data into one generalist policy. It improves robots on difficult long-horizon tasks. Great release by Robbyant.

elvis

10,464 次观看 • 4 个月前

Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced

Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced

Chen Wang

239,752 次观看 • 2 年前

X Square Robot has introduced TwinDEX: a co-designed pair of three-finger, 9-DoF dexterous hands, one wearable for data collection, one matched robot end effector for deployment. The wearable and the robot hand share the same kinematics, geometry, materials, look, and sensors, so human demos map straight to the robot without retargeting. Human data collection improves throughput 5.3x compared to teleoperation, and a policy can be trained with a few hundred episodes without any robot data.

X Square Robot has introduced TwinDEX: a co-designed pair of three-finger, 9-DoF dexterous hands, one wearable for data collection, one matched robot end effector for deployment. The wearable and the robot hand share the same kinematics, geometry, materials, look, and sensors, so human demos map straight to the robot without retargeting. Human data collection improves throughput 5.3x compared to teleoperation, and a policy can be trained with a few hundred episodes without any robot data.

The Humanoid Hub

277,022 次观看 • 2 个月前

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes evencheaper with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,358 次观看 • 1 年前

Egocentric Videos for Robot Training 🧵 Researchers at NYU and UC Berkeley published research where they developed a system called EgoZero that trains robots using human demonstration videos recorded with smart glasses. 👓 The system converts first-person human actions into 3D point-based state-action representations. The policies executed on a gripper-equipped robot, achieved a 70% zero-shot success rate across seven manipulation tasks, with only 20 minutes of human data per task. EgoZero stands as one of the solid empirical proofs that egocentric smart-glass video collected from everyday human behavior can serve as powerful, scalable training data for real robot learning. Vincent Liu Ademi Adeniji

Egocentric Videos for Robot Training 🧵 Researchers at NYU and UC Berkeley published research where they developed a system called EgoZero that trains robots using human demonstration videos recorded with smart glasses. 👓 The system converts first-person human actions into 3D point-based state-action representations. The policies executed on a gripper-equipped robot, achieved a 70% zero-shot success rate across seven manipulation tasks, with only 20 minutes of human data per task. EgoZero stands as one of the solid empirical proofs that egocentric smart-glass video collected from everyday human behavior can serve as powerful, scalable training data for real robot learning. Vincent Liu Ademi Adeniji

VaderResearch

23,055 次观看 • 1 年前

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 次观看 • 1 年前

Tired of collecting demonstrations all day to train your robot? Introducing MimicGen, an autonomous data generation system for robotics. Using just 200 human demos we generated a large multi-task dataset of 50K demos! #CoRL2023 #NVIDIAResearch 👇 🧵 1/

Tired of collecting demonstrations all day to train your robot? Introducing MimicGen, an autonomous data generation system for robotics. Using just 200 human demos we generated a large multi-task dataset of 50K demos! #CoRL2023 #NVIDIAResearch 👇 🧵 1/

Ajay Mandlekar

93,696 次观看 • 3 年前

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵

Mahi Shafiullah 🛫 CoRL 🏠🤖

102,160 次观看 • 4 个月前

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Max Fu

40,466 次观看 • 2 年前

Happy to share what I’ve been working on since joining Genesis! GENE-26.5 is a one-of-a-kind, robotics-native multimodal foundation model that learns from diverse, in-the-wild data across modalities and outputs actions enabling a 54-DoF robot system to perform the most dexterous, long-horizon manipulation tasks to date—approaching human-level capability. This is the result of innovations across the full stack—data collection and processing, robot systems, model architecture, training strategies, and scalable evaluation infrastructure.

Happy to share what I’ve been working on since joining Genesis! GENE-26.5 is a one-of-a-kind, robotics-native multimodal foundation model that learns from diverse, in-the-wild data across modalities and outputs actions enabling a 54-DoF robot system to perform the most dexterous, long-horizon manipulation tasks to date—approaching human-level capability. This is the result of innovations across the full stack—data collection and processing, robot systems, model architecture, training strategies, and scalable evaluation infrastructure.

Zu

20,615 次观看 • 6 个月前