Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Can we learn whole-body mobile manipulation directly from human demonstrations? Introducing Whole-Body Mobile Manipulation Interface (HoMMI) Egocentric + UMI, 0 teleop -> bimanual & whole-body manipulation, long-horizon navigation, active perception

Xiaomeng Xu

1,259 subscribers

82,474 views • 4 months ago •via X (Twitter)

Education Health & Wellness Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Autonomous end to end whole body mobile manipulation, via Noble Machines. All one policy.

Autonomous end to end whole body mobile manipulation, via Noble Machines. All one policy.

Chris Paxton

12,023 views • 5 months ago

Can we collect robot data without any robots? Introducing Universal Manipulation Interface (UMI) An open-source $400 system from Stanford University designed to democratize robot data collection 0 teleop -> autonomously wash dishes (precise), toss (dynamic), and fold clothes (bimanual)

Can we collect robot data without any robots? Introducing Universal Manipulation Interface (UMI) An open-source $400 system from Stanford University designed to democratize robot data collection 0 teleop -> autonomously wash dishes (precise), toss (dynamic), and fold clothes (bimanual)

Cheng Chi

439,417 views • 2 years ago

Its been doing this for like two hours. RL whole body mobile manipulation, objects bought on Monday, never seen before

Its been doing this for like two hours. RL whole body mobile manipulation, objects bought on Monday, never seen before

Chris Paxton

43,535 views • 1 year ago

Whole-body Large Behavior Model diffusion policy from Boston Dynamics and TRI. Most advanced mobile-manipulation demo with error recovery I've seen to date.

Whole-body Large Behavior Model diffusion policy from Boston Dynamics and TRI. Most advanced mobile-manipulation demo with error recovery I've seen to date.

Simon Kalouche

41,706 views • 11 months ago

X-Humanoid unveiled TG-VLA, which it calls the world's first whole-body VLA framework for full-size humanoid robots. The goal: move humanoids past "mobile dual-arm systems" to human-like, whole-body coordinated operation. Built on three technologies: - HEX: an open-source full-size humanoid VLA that learns across embodiments and coordinates the whole body. Claimed SOTA on real-world whole-body manipulation - HAF-VLA: hierarchical action flow that decomposes whole-body motion into subtasks. Best or joint-best on 6 benchmarks - DSRL-DCT: online RL for high-DoF humanoids in a compressed latent space. 100% success on mobile manipulation HEX is open-source.

X-Humanoid unveiled TG-VLA, which it calls the world's first whole-body VLA framework for full-size humanoid robots. The goal: move humanoids past "mobile dual-arm systems" to human-like, whole-body coordinated operation. Built on three technologies: - HEX: an open-source full-size humanoid VLA that learns across embodiments and coordinates the whole body. Claimed SOTA on real-world whole-body manipulation - HAF-VLA: hierarchical action flow that decomposes whole-body motion into subtasks. Best or joint-best on 6 benchmarks - DSRL-DCT: online RL for high-DoF humanoids in a compressed latent space. 100% success on mobile manipulation HEX is open-source.

The Humanoid Hub

19,563 views • 1 month ago

🤖 Introducing ω-0 (OMEGA-0) — a whole-body World Action Model for humanoids. Can one model make a humanoid walk, manipulate, and coordinate its whole body simultaneously across many real-world tasks? ω-0 does exactly that. ⚡ One unified model for multi-task whole-body loco-manipulation 🧠 Predicts future visual latents while generating executable whole-body actions 🏠 Trained with ω-HOME: 40+ hours, 24 household tasks, 4,827 episodes 🔥 81.8% real-world success rate across 11 evaluation tasks No separate locomotion/manipulation policies. One model. Whole body. Multiple tasks. Humanoid intelligence should not be a collection of isolated skills. It should be a unified whole-body policy. 🔗 📜 One-take, uncut real-world demo👇

🤖 Introducing ω-0 (OMEGA-0) — a whole-body World Action Model for humanoids. Can one model make a humanoid walk, manipulate, and coordinate its whole body simultaneously across many real-world tasks? ω-0 does exactly that. ⚡ One unified model for multi-task whole-body loco-manipulation 🧠 Predicts future visual latents while generating executable whole-body actions 🏠 Trained with ω-HOME: 40+ hours, 24 household tasks, 4,827 episodes 🔥 81.8% real-world success rate across 11 evaluation tasks No separate locomotion/manipulation policies. One model. Whole body. Multiple tasks. Humanoid intelligence should not be a collection of isolated skills. It should be a unified whole-body policy. 🔗 📜 One-take, uncut real-world demo👇

Jianfei Yang

14,836 views • 6 days ago

Introducing the QUANXTA Zero Series, X Square Robot’s next-generation UMI data collection solution for embodied AI. The series includes three products: QUANXTA Zero-G0: VR headset + backpack + dual grippers QUANXTA Zero-G1: headband rig + dual grippers QUANXTA Zero-E0: headband rig Built for scalable embodied AI data production: Multimodal capture. Whole-body mobile manipulation. Real-robot replay. 1 ms synchronization. 100% frame-level alignment. From human demonstrations to trainable robot data. Learn more:

Introducing the QUANXTA Zero Series, X Square Robot’s next-generation UMI data collection solution for embodied AI. The series includes three products: QUANXTA Zero-G0: VR headset + backpack + dual grippers QUANXTA Zero-G1: headband rig + dual grippers QUANXTA Zero-E0: headband rig Built for scalable embodied AI data production: Multimodal capture. Whole-body mobile manipulation. Real-robot replay. 1 ms synchronization. 100% frame-level alignment. From human demonstrations to trainable robot data. Learn more:

X Square Robot

354,518 views • 1 month ago

Real-Time Omni-Modal Interaction Driven Whole-Body Mobile Manipulation🥳 Unitree UnifoLM-OminiA-0.3 — a single model handling diverse home-care and wellness tasks, with omni-modal interactive understanding, fully autonomous, stable and disturbance-resistant execution.

Real-Time Omni-Modal Interaction Driven Whole-Body Mobile Manipulation🥳 Unitree UnifoLM-OminiA-0.3 — a single model handling diverse home-care and wellness tasks, with omni-modal interactive understanding, fully autonomous, stable and disturbance-resistant execution.

Unitree

15,477,044 views • 26 days ago

Data is the core bottleneck for robotics, here is the solution: Introducing MicroAGI00: We open sourced 1M+ frames of egocentric human long horizon mobile manipulation data. Scaling to 300M by Dec '25 and 100B frames Q4 2026 dm if you want your robot to wipe your office 🧵

Data is the core bottleneck for robotics, here is the solution: Introducing MicroAGI00: We open sourced 1M+ frames of egocentric human long horizon mobile manipulation data. Scaling to 300M by Dec '25 and 100B frames Q4 2026 dm if you want your robot to wipe your office 🧵

Bercan

39,646 views • 10 months ago

The table-to-dishwasher task is the classic nightmare scenario for roboticists: Long-horizon, highly dexterous, precise, whole-body manipulation combined with delicate, transparent, reflective, and deformable objects. Yet Memo handles it so naturally and elegantly.

The table-to-dishwasher task is the classic nightmare scenario for roboticists: Long-horizon, highly dexterous, precise, whole-body manipulation combined with delicate, transparent, reflective, and deformable objects. Yet Memo handles it so naturally and elegantly.

Tony Zhao

802,304 views • 8 months ago

🤖 How can we scale up humanoid robot learning? Introducing 🌟VLK🌟: generating large-scale synthetic data with paired egocentric observations, text, and full-body G1 kinematics for learning humanoid loco-manipulation. No teleoperation needed! Website:

🤖 How can we scale up humanoid robot learning? Introducing 🌟VLK🌟: generating large-scale synthetic data with paired egocentric observations, text, and full-body G1 kinematics for learning humanoid loco-manipulation. No teleoperation needed! Website:

Jiaman Li

971,972 views • 1 month ago

Okay, this is genuinely impressive. 🤯 Oli just worked through a roomful of household chores — no teleop, no cuts, one continuous take. The brain is LimX COSA 0.5, a three-layer stack coordinating reasoning (Sys 2), skill execution (Sys 1), and motion control (Sys 0), along with memory, task scheduling, and whole-body coordination. What matters isn’t one polished demo trick. It’s that the robot stayed coherent across an entire long-horizon mobile manipulation sequence — on real hardware, in one take.

Okay, this is genuinely impressive. 🤯 Oli just worked through a roomful of household chores — no teleop, no cuts, one continuous take. The brain is LimX COSA 0.5, a three-layer stack coordinating reasoning (Sys 2), skill execution (Sys 1), and motion control (Sys 0), along with memory, task scheduling, and whole-body coordination. What matters isn’t one polished demo trick. It’s that the robot stayed coherent across an entire long-horizon mobile manipulation sequence — on real hardware, in one take.

RoboHub🤖

33,607 views • 1 month ago

Boston Dynamics’ Spot Masters Whole-Body Manipulation — Using AI to Drag, Roll, and Stack 15 kg Tires with Precision by RAI Institute #Robotics #Engineering #ArtificialIntelligence #Innovation #Technology

Boston Dynamics’ Spot Masters Whole-Body Manipulation — Using AI to Drag, Roll, and Stack 15 kg Tires with Precision by RAI Institute #Robotics #Engineering #ArtificialIntelligence #Innovation #Technology

Ronald van Loon

39,538 views • 7 months ago

See Spot perform dynamic whole-body manipulation. Using a combination of reinforcement learning (RL) and sampling-based control, the robot is able to autonomously drag, roll, and stack tires weighing 15 kg (33 lb), well above its maximum arm lift capacity. Learn more about coordinating locomotion and manipulation processes:

See Spot perform dynamic whole-body manipulation. Using a combination of reinforcement learning (RL) and sampling-based control, the robot is able to autonomously drag, roll, and stack tires weighing 15 kg (33 lb), well above its maximum arm lift capacity. Learn more about coordinating locomotion and manipulation processes:

RAI Institute

87,583 views • 10 months ago

LimX just released the TRON 2 Autonomous Mobile Manipulation Kit, and demonstrated the possibilities of modular design. One use case demo I really like shows dexterous hand picking up takeout, the wheeled-legged robot smoothly climbing a flight of stairs, and then handing it off to another TRON2 It combines the speed of wheels with the high mobility of legs,3D perception, autonomous navigation, whole-body motion control+manipulation… --- Not every building has an elevator. A wheeled-legged TRON 2 could become a truly reliable life assistant, especially for elderly people living alone.

LimX just released the TRON 2 Autonomous Mobile Manipulation Kit, and demonstrated the possibilities of modular design. One use case demo I really like shows dexterous hand picking up takeout, the wheeled-legged robot smoothly climbing a flight of stairs, and then handing it off to another TRON2 It combines the speed of wheels with the high mobility of legs,3D perception, autonomous navigation, whole-body motion control+manipulation… --- Not every building has an elevator. A wheeled-legged TRON 2 could become a truly reliable life assistant, especially for elderly people living alone.

CyberRobo

39,514 views • 26 days ago

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

Mandi Zhao

121,265 views • 1 year ago

🤖 Ever wondered what robots need to truly help humans around the house? 🏡 Introducing 𝗕𝗘𝗛𝗔𝗩𝗜𝗢𝗥 𝗥𝗼𝗯𝗼𝘁 𝗦𝘂𝗶𝘁𝗲 (𝗕𝗥𝗦)—a comprehensive framework for mastering mobile whole-body manipulation across diverse household tasks! 🧹🫧 From taking out the trash to laying out clothes and cleaning toilets—𝗕𝗥𝗦 equips robots to handle practical, everyday activities. 🌐 Explore more: Let's dive in! 🤿🧵

🤖 Ever wondered what robots need to truly help humans around the house? 🏡 Introducing 𝗕𝗘𝗛𝗔𝗩𝗜𝗢𝗥 𝗥𝗼𝗯𝗼𝘁 𝗦𝘂𝗶𝘁𝗲 (𝗕𝗥𝗦)—a comprehensive framework for mastering mobile whole-body manipulation across diverse household tasks! 🧹🫧 From taking out the trash to laying out clothes and cleaning toilets—𝗕𝗥𝗦 equips robots to handle practical, everyday activities. 🌐 Explore more: Let's dive in! 🤿🧵

Yunfan Jiang

186,173 views • 1 year ago

NEWS: Humanoid robotics company Figure has released Helix 02, what they claim in their most capable humanoid model yet. "A single neural system that controls the full body directly from pixels, enabling dexterous, long horizon autonomy across an entire room: • Autonomous, long‑horizon loco-manipulation: Helix 02 unloads and reloads a dishwasher across a full-sized kitchen - a four-minute, end-to-end autonomous task that integrates walking, manipulation, and balance with no resets and no human intervention. We believe this is the longest horizon, most complex task completed autonomously by a humanoid robot to date. • All sensors in. All actuators out: Helix 02 connects every onboard sensor - vision, touch, and proprioception - directly to every actuator through a single unified visuomotor neural network. • Human-like whole body control from human data: All results are enabled by System 0, a learned whole‑body controller trained on over 1,000 hours of human motion data and sim‑to‑real reinforcement learning. System 0 replaces 109,504 lines of hand‑engineered C++ with a single neural prior for stable, natural motion. • New classes of dexterity: With Figure 03’s embedded tactile sensing and palm cameras, Helix 02 performs manipulation that was previously out of reach: extracting individual pills, dispensing precise syringe volumes, and singulating small, irregular objects from clutter despite self‑occlusion. Helix 02 is trained on over 1,000 hours of human motion data and integrates vision, touch, and proprioception."

NEWS: Humanoid robotics company Figure has released Helix 02, what they claim in their most capable humanoid model yet. "A single neural system that controls the full body directly from pixels, enabling dexterous, long horizon autonomy across an entire room: • Autonomous, long‑horizon loco-manipulation: Helix 02 unloads and reloads a dishwasher across a full-sized kitchen - a four-minute, end-to-end autonomous task that integrates walking, manipulation, and balance with no resets and no human intervention. We believe this is the longest horizon, most complex task completed autonomously by a humanoid robot to date. • All sensors in. All actuators out: Helix 02 connects every onboard sensor - vision, touch, and proprioception - directly to every actuator through a single unified visuomotor neural network. • Human-like whole body control from human data: All results are enabled by System 0, a learned whole‑body controller trained on over 1,000 hours of human motion data and sim‑to‑real reinforcement learning. System 0 replaces 109,504 lines of hand‑engineered C++ with a single neural prior for stable, natural motion. • New classes of dexterity: With Figure 03’s embedded tactile sensing and palm cameras, Helix 02 performs manipulation that was previously out of reach: extracting individual pills, dispensing precise syringe volumes, and singulating small, irregular objects from clutter despite self‑occlusion. Helix 02 is trained on over 1,000 hours of human motion data and integrates vision, touch, and proprioception."

Sawyer Merritt

624,770 views • 6 months ago

Just collecting manipulation data isn’t enough for robots - they need to be able to move around in the world, which has a whole different set of challenges from pure manipulation. And bringing navigation and manipulation together in a single framework is even more challenging. Enter HERMES, from Zhecheng Yuan and Tianming Wei. This is a four-stage process in which human videos are used to set up an RL sim-to-real training pipeline in order to overcome differences between robot and human kinematics, and used together with a navigation foundation model to move around in a variety of environments. To learn more, join us as Zhecheng Yuan and Tianming Wei tell us about how they built their system to perform mobile dexterous manipulation from human videos in a variety of environments. Watch Episode #45 of RoboPapers today, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

Just collecting manipulation data isn’t enough for robots - they need to be able to move around in the world, which has a whole different set of challenges from pure manipulation. And bringing navigation and manipulation together in a single framework is even more challenging. Enter HERMES, from Zhecheng Yuan and Tianming Wei. This is a four-stage process in which human videos are used to set up an RL sim-to-real training pipeline in order to overcome differences between robot and human kinematics, and used together with a navigation foundation model to move around in a variety of environments. To learn more, join us as Zhecheng Yuan and Tianming Wei tell us about how they built their system to perform mobile dexterous manipulation from human videos in a variety of environments. Watch Episode #45 of RoboPapers today, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

15,092 views • 8 months ago

Mobile-TeleVision follows our previous idea on upper-body and lower-body separation for control: 1⃣We train RL for lower-body locomotion and standing; 2⃣We use IK and retargeting directly for upper-body for high-precision and smooth manipulation. We avoid using RL for upper-body motion tracking. This is because fundamentally it is hard to avoid errors with RL tracking, and sim2real gap brings another gap. At this point, we still find using IK is more practical and precise.

Mobile-TeleVision follows our previous idea on upper-body and lower-body separation for control: 1⃣We train RL for lower-body locomotion and standing; 2⃣We use IK and retargeting directly for upper-body for high-precision and smooth manipulation. We avoid using RL for upper-body motion tracking. This is because fundamentally it is hard to avoid errors with RL tracking, and sim2real gap brings another gap. At this point, we still find using IK is more practical and precise.

Xiaolong Wang

32,662 views • 1 year ago