Loading video...

Video Failed to Load

Go Home

HiFi-UMI Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone paper:

_akhaliq's profile picture

AK

514,711 subscribers

46,235 views • 2 days ago •via X (Twitter)

Science & TechnologyEducation
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

UMI-style robot learning with mobile manipulation and multiple robots
2:51

UMI-style robot learning with mobile manipulation and multiple robots

Chris Paxton

12,148 views • 1 month ago

HACK: Learning a Parametric Head and Neck Model for High-fidelity Animation abs: paper page:
2:41

HACK: Learning a Parametric Head and Neck Model for High-fidelity Animation abs: paper page:

AK

90,156 views • 3 years ago

Can we collect robot data without any robots? Introducing Universal Manipulation Interface (UMI) An open-source $400 system from Stanford University designed to democratize robot data collection 0 teleop -> autonomously wash dishes (precise), toss (dynamic), and fold clothes (bimanual)
1:16

Can we collect robot data without any robots? Introducing Universal Manipulation Interface (UMI) An open-source $400 system from Stanford University designed to democratize robot data collection 0 teleop -> autonomously wash dishes (precise), toss (dynamic), and fold clothes (bimanual)

Cheng Chi

439,244 views • 2 years ago

Data can’t just be outsourced🤯 To iterate fast, robotics teams must own their data infrastructure Introducing SyncField: turnkey data infrastructure for in-the-wild data collection (Best for UMI-style & Embodied human) #Robotics #UMI #DataCollection
1:40

Data can’t just be outsourced🤯 To iterate fast, robotics teams must own their data infrastructure Introducing SyncField: turnkey data infrastructure for in-the-wild data collection (Best for UMI-style & Embodied human) #Robotics #UMI #DataCollection

Julia Kim @ICML

15,055 views • 4 months ago

Can we learn whole-body mobile manipulation directly from human demonstrations? Introducing Whole-Body Mobile Manipulation Interface (HoMMI) Egocentric + UMI, 0 teleop -> bimanual & whole-body manipulation, long-horizon navigation, active perception
0:28

Can we learn whole-body mobile manipulation directly from human demonstrations? Introducing Whole-Body Mobile Manipulation Interface (HoMMI) Egocentric + UMI, 0 teleop -> bimanual & whole-body manipulation, long-horizon navigation, active perception

Xiaomeng Xu

82,042 views • 4 months ago

A HUG FROM UMI COULD HEAL ANYONE 🥹
0:46

A HUG FROM UMI COULD HEAL ANYONE 🥹

𝐌𝐢𝐥𝐤 ✿

13,774 views • 1 month ago

Are all robots equally UMI-able? New research from CMU Robotics Institute + Stanford AI Lab : UMI-on-Air proves that they can be .
0:20

Are all robots equally UMI-able? New research from CMU Robotics Institute + Stanford AI Lab : UMI-on-Air proves that they can be .

Robots Digest 🤖

31,330 views • 8 months ago

Vision alone isn’t enough to solve dexterous manipulation. The sense of touch is needed. UMI-FT uses a custom PCB to provide 6-axis force-torque sensing at each finger for under $10.
0:21

Vision alone isn’t enough to solve dexterous manipulation. The sense of touch is needed. UMI-FT uses a custom PCB to provide 6-axis force-torque sensing at each finger for under $10.

Simon Kalouche

20,634 views • 6 months ago

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)
0:33

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

Chris Paxton

20,486 views • 1 year ago

Data Collector uses our General Intelligence Labs UMI grippers to do welding at a construction site ( video credit: ghz 🏴‍☠️ )
1:23

Data Collector uses our General Intelligence Labs UMI grippers to do welding at a construction site ( video credit: ghz 🏴‍☠️ )

Shumo Chu

31,681 views • 2 months ago

Imitation learning works™ – but you need good data 🥹 How to get high-quality visuotactile demos from a bimanual robot with multifingered hands, and learn smooth policies? Check our new work “Learning Visuotactile Skills with Two Multifingered Hands”! 🙌
0:29

Imitation learning works™ – but you need good data 🥹 How to get high-quality visuotactile demos from a bimanual robot with multifingered hands, and learn smooth policies? Check our new work “Learning Visuotactile Skills with Two Multifingered Hands”! 🙌

Toru

61,504 views • 2 years ago

Aside from protecting Yuu’s reputation as the class idol, let’s be honest—Umi is doing absolutely everything she can to stop Yuu from spending any time alone with Maki... #クラにか
1:31

Aside from protecting Yuu’s reputation as the class idol, let’s be honest—Umi is doing absolutely everything she can to stop Yuu from spending any time alone with Maki... #クラにか

𝓡𝓲𝔁𝓲 🌺

23,503 views • 3 months ago

Tactile feedback is one of the most important modalities in manipulation, but has been underutilized in dexterous hands. T-Dex is a framework for learning dexterous policies from tactile play data, beating vision and torque-based methods by 1.7x. 🧵👇
0:55

Tactile feedback is one of the most important modalities in manipulation, but has been underutilized in dexterous hands. T-Dex is a framework for learning dexterous policies from tactile play data, beating vision and torque-based methods by 1.7x. 🧵👇

Lerrel Pinto

84,611 views • 3 years ago

BAEKHYUN, UMI, EL CAPITXN have released 'Do What You Do', rate it from 1 to 10!
0:27

BAEKHYUN, UMI, EL CAPITXN have released 'Do What You Do', rate it from 1 to 10!

About Music

361,388 views • 1 year ago

Introducing RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning. 7 real robot tasks, 900/900 successes. Up to 250 consecutive trials in one task, running 2 hours nonstop without failure. High success rate against physical disturbances, zero-shot, and few-shot adaptation Our first step toward a deployable robot learning system.
3:26

Introducing RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning. 7 real robot tasks, 900/900 successes. Up to 250 consecutive trials in one task, running 2 hours nonstop without failure. High success rate against physical disturbances, zero-shot, and few-shot adaptation Our first step toward a deployable robot learning system.

Kun Lei

91,213 views • 9 months ago

Introducing the QUANXTA Zero Series, X Square Robot’s next-generation UMI data collection solution for embodied AI. The series includes three products: QUANXTA Zero-G0: VR headset + backpack + dual grippers QUANXTA Zero-G1: headband rig + dual grippers QUANXTA Zero-E0: headband rig Built for scalable embodied AI data production: Multimodal capture. Whole-body mobile manipulation. Real-robot replay. 1 ms synchronization. 100% frame-level alignment. From human demonstrations to trainable robot data. Learn more:
2:29

Introducing the QUANXTA Zero Series, X Square Robot’s next-generation UMI data collection solution for embodied AI. The series includes three products: QUANXTA Zero-G0: VR headset + backpack + dual grippers QUANXTA Zero-G1: headband rig + dual grippers QUANXTA Zero-E0: headband rig Built for scalable embodied AI data production: Multimodal capture. Whole-body mobile manipulation. Real-robot replay. 1 ms synchronization. 100% frame-level alignment. From human demonstrations to trainable robot data. Learn more:

X Square Robot

354,518 views • 25 days ago

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork
0:20

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

Pascal Bornet

186,223 views • 3 months ago

Amazon is training humanoids to move boxes. Makes sense! OmniRetarget is a data generation engine that enables complex loco-manipulation for humanoids. It uses offline retargeting from human MoCap datasets and augments data from single demos to produce 8 hours of trajectories that train RL policies in simulation, transferred zero-shot to real robots. Amazon FAR is the primary affiliation for the paper:
1:37

Amazon is training humanoids to move boxes. Makes sense! OmniRetarget is a data generation engine that enables complex loco-manipulation for humanoids. It uses offline retargeting from human MoCap datasets and augments data from single demos to produce 8 hours of trajectories that train RL policies in simulation, transferred zero-shot to real robots. Amazon FAR is the primary affiliation for the paper:

The Humanoid Hub

106,447 views • 10 months ago

If you notice, Umi has actually started to teasing Maki in ways that let her interact more closely and intimate with him 🩷 And as time goes on, we'll be getting more and more of this kind of teasing from Umi 🥹 #クラにか
1:37

If you notice, Umi has actually started to teasing Maki in ways that let her interact more closely and intimate with him 🩷 And as time goes on, we'll be getting more and more of this kind of teasing from Umi 🥹 #クラにか

𝓡𝓲𝔁𝓲 🌺

43,624 views • 2 months ago