Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Can we learn whole-body mobile manipulation directly from human demonstrations? Introducing Whole-Body Mobile Manipulation Interface (HoMMI) Egocentric + UMI, 0 teleop -> bimanual & whole-body manipulation, long-horizon navigation, active perception

XiaomengXu11's profile picture

Xiaomeng Xu

1,187 subscribers

75,955 görüntüleme • 3 ay önce •via X (Twitter)

EğitimSağlık & İyilikBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Autonomous end to end whole body mobile manipulation, via Noble Machines. All one policy.
0:56

Autonomous end to end whole body mobile manipulation, via Noble Machines. All one policy.

Chris Paxton

12,023 görüntüleme • 3 ay önce

Can we collect robot data without any robots? Introducing Universal Manipulation Interface (UMI) An open-source $400 system from Stanford University designed to democratize robot data collection 0 teleop -> autonomously wash dishes (precise), toss (dynamic), and fold clothes (bimanual)
1:16

Can we collect robot data without any robots? Introducing Universal Manipulation Interface (UMI) An open-source $400 system from Stanford University designed to democratize robot data collection 0 teleop -> autonomously wash dishes (precise), toss (dynamic), and fold clothes (bimanual)

Cheng Chi

438,741 görüntüleme • 2 yıl önce

Its been doing this for like two hours. RL whole body mobile manipulation, objects bought on Monday, never seen before
1:18

Its been doing this for like two hours. RL whole body mobile manipulation, objects bought on Monday, never seen before

Chris Paxton

43,535 görüntüleme • 1 yıl önce

Whole-body Large Behavior Model diffusion policy from Boston Dynamics and TRI. Most advanced mobile-manipulation demo with error recovery I've seen to date.
0:22

Whole-body Large Behavior Model diffusion policy from Boston Dynamics and TRI. Most advanced mobile-manipulation demo with error recovery I've seen to date.

Simon Kalouche

41,706 görüntüleme • 10 ay önce

Introduce OmniH2O, a learning-based system for whole-body humanoid teleop and autonomy: 🦾Robust loco-mani policy 🦸Universal teleop interface: VR, verbal, RGB 🧠Autonomy via ShitposterAI or imitation 🔗Release the first whole-body humanoid dataset
2:06

Introduce OmniH2O, a learning-based system for whole-body humanoid teleop and autonomy: 🦾Robust loco-mani policy 🦸Universal teleop interface: VR, verbal, RGB 🧠Autonomy via ShitposterAI or imitation 🔗Release the first whole-body humanoid dataset

Tairan He

130,032 görüntüleme • 2 yıl önce

Data is the core bottleneck for robotics, here is the solution: Introducing MicroAGI00: We open sourced 1M+ frames of egocentric human long horizon mobile manipulation data. Scaling to 300M by Dec '25 and 100B frames Q4 2026 dm if you want your robot to wipe your office 🧵
0:29

Data is the core bottleneck for robotics, here is the solution: Introducing MicroAGI00: We open sourced 1M+ frames of egocentric human long horizon mobile manipulation data. Scaling to 300M by Dec '25 and 100B frames Q4 2026 dm if you want your robot to wipe your office 🧵

Bercan

39,646 görüntüleme • 8 ay önce

The table-to-dishwasher task is the classic nightmare scenario for roboticists: Long-horizon, highly dexterous, precise, whole-body manipulation combined with delicate, transparent, reflective, and deformable objects. Yet Memo handles it so naturally and elegantly.
1:18

The table-to-dishwasher task is the classic nightmare scenario for roboticists: Long-horizon, highly dexterous, precise, whole-body manipulation combined with delicate, transparent, reflective, and deformable objects. Yet Memo handles it so naturally and elegantly.

Tony Zhao

799,314 görüntüleme • 7 ay önce

Boston Dynamics’ Spot Masters Whole-Body Manipulation — Using AI to Drag, Roll, and Stack 15 kg Tires with Precision by RAI Institute #Robotics #Engineering #ArtificialIntelligence #Innovation #Technology
0:27

Boston Dynamics’ Spot Masters Whole-Body Manipulation — Using AI to Drag, Roll, and Stack 15 kg Tires with Precision by RAI Institute #Robotics #Engineering #ArtificialIntelligence #Innovation #Technology

Ronald van Loon

39,538 görüntüleme • 6 ay önce

See Spot perform dynamic whole-body manipulation. Using a combination of reinforcement learning (RL) and sampling-based control, the robot is able to autonomously drag, roll, and stack tires weighing 15 kg (33 lb), well above its maximum arm lift capacity. Learn more about coordinating locomotion and manipulation processes:
0:27

See Spot perform dynamic whole-body manipulation. Using a combination of reinforcement learning (RL) and sampling-based control, the robot is able to autonomously drag, roll, and stack tires weighing 15 kg (33 lb), well above its maximum arm lift capacity. Learn more about coordinating locomotion and manipulation processes:

RAI Institute

87,430 görüntüleme • 8 ay önce

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.
0:43

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

Mandi Zhao

120,304 görüntüleme • 1 yıl önce

🤖 Ever wondered what robots need to truly help humans around the house? 🏡 Introducing 𝗕𝗘𝗛𝗔𝗩𝗜𝗢𝗥 𝗥𝗼𝗯𝗼𝘁 𝗦𝘂𝗶𝘁𝗲 (𝗕𝗥𝗦)—a comprehensive framework for mastering mobile whole-body manipulation across diverse household tasks! 🧹🫧 From taking out the trash to laying out clothes and cleaning toilets—𝗕𝗥𝗦 equips robots to handle practical, everyday activities. 🌐 Explore more: Let's dive in! 🤿🧵
1:09

🤖 Ever wondered what robots need to truly help humans around the house? 🏡 Introducing 𝗕𝗘𝗛𝗔𝗩𝗜𝗢𝗥 𝗥𝗼𝗯𝗼𝘁 𝗦𝘂𝗶𝘁𝗲 (𝗕𝗥𝗦)—a comprehensive framework for mastering mobile whole-body manipulation across diverse household tasks! 🧹🫧 From taking out the trash to laying out clothes and cleaning toilets—𝗕𝗥𝗦 equips robots to handle practical, everyday activities. 🌐 Explore more: Let's dive in! 🤿🧵

Yunfan Jiang

185,838 görüntüleme • 1 yıl önce

Your bimanual manipulators might need a Robot Neck 🤖🦒 Introducing Vision in Action: Learning Active Perception from Human Demonstrations ViA learns task-specific, active perceptual strategies—such as searching, tracking, and focusing—directly from human demos, enabling robust visuomotor policies under visual occlusions. 🧵👇
1:25

Your bimanual manipulators might need a Robot Neck 🤖🦒 Introducing Vision in Action: Learning Active Perception from Human Demonstrations ViA learns task-specific, active perceptual strategies—such as searching, tracking, and focusing—directly from human demos, enabling robust visuomotor policies under visual occlusions. 🧵👇

Haoyu Xiong

122,084 görüntüleme • 1 yıl önce

Can robots leverage their entire body to sense and interact with their environment, rather than just relying on a centralized camera and end-effector? Introducing RoboPanoptes, a robot system that achieves whole-body dexterity through whole-body vision.
0:34

Can robots leverage their entire body to sense and interact with their environment, rather than just relying on a centralized camera and end-effector? Introducing RoboPanoptes, a robot system that achieves whole-body dexterity through whole-body vision.

Xiaomeng Xu

76,351 görüntüleme • 1 yıl önce

NEWS: Humanoid robotics company Figure has released Helix 02, what they claim in their most capable humanoid model yet. "A single neural system that controls the full body directly from pixels, enabling dexterous, long horizon autonomy across an entire room: • Autonomous, long‑horizon loco-manipulation: Helix 02 unloads and reloads a dishwasher across a full-sized kitchen - a four-minute, end-to-end autonomous task that integrates walking, manipulation, and balance with no resets and no human intervention. We believe this is the longest horizon, most complex task completed autonomously by a humanoid robot to date. • All sensors in. All actuators out: Helix 02 connects every onboard sensor - vision, touch, and proprioception - directly to every actuator through a single unified visuomotor neural network. • Human-like whole body control from human data: All results are enabled by System 0, a learned whole‑body controller trained on over 1,000 hours of human motion data and sim‑to‑real reinforcement learning. System 0 replaces 109,504 lines of hand‑engineered C++ with a single neural prior for stable, natural motion. • New classes of dexterity: With Figure 03’s embedded tactile sensing and palm cameras, Helix 02 performs manipulation that was previously out of reach: extracting individual pills, dispensing precise syringe volumes, and singulating small, irregular objects from clutter despite self‑occlusion. Helix 02 is trained on over 1,000 hours of human motion data and integrates vision, touch, and proprioception."
3:25

NEWS: Humanoid robotics company Figure has released Helix 02, what they claim in their most capable humanoid model yet. "A single neural system that controls the full body directly from pixels, enabling dexterous, long horizon autonomy across an entire room: • Autonomous, long‑horizon loco-manipulation: Helix 02 unloads and reloads a dishwasher across a full-sized kitchen - a four-minute, end-to-end autonomous task that integrates walking, manipulation, and balance with no resets and no human intervention. We believe this is the longest horizon, most complex task completed autonomously by a humanoid robot to date. • All sensors in. All actuators out: Helix 02 connects every onboard sensor - vision, touch, and proprioception - directly to every actuator through a single unified visuomotor neural network. • Human-like whole body control from human data: All results are enabled by System 0, a learned whole‑body controller trained on over 1,000 hours of human motion data and sim‑to‑real reinforcement learning. System 0 replaces 109,504 lines of hand‑engineered C++ with a single neural prior for stable, natural motion. • New classes of dexterity: With Figure 03’s embedded tactile sensing and palm cameras, Helix 02 performs manipulation that was previously out of reach: extracting individual pills, dispensing precise syringe volumes, and singulating small, irregular objects from clutter despite self‑occlusion. Helix 02 is trained on over 1,000 hours of human motion data and integrates vision, touch, and proprioception."

Sawyer Merritt

624,668 görüntüleme • 5 ay önce

Introducing ClickDiffusion! We developed a system for precise image manipulation and generation that combines natural language instructions with visual feedback provided by the user through a direct manipulation interface.
0:30

Introducing ClickDiffusion! We developed a system for precise image manipulation and generation that combines natural language instructions with visual feedback provided by the user through a direct manipulation interface.

Alec Helbling

36,296 görüntüleme • 2 yıl önce

Just collecting manipulation data isn’t enough for robots - they need to be able to move around in the world, which has a whole different set of challenges from pure manipulation. And bringing navigation and manipulation together in a single framework is even more challenging. Enter HERMES, from Zhecheng Yuan and Tianming Wei. This is a four-stage process in which human videos are used to set up an RL sim-to-real training pipeline in order to overcome differences between robot and human kinematics, and used together with a navigation foundation model to move around in a variety of environments. To learn more, join us as Zhecheng Yuan and Tianming Wei tell us about how they built their system to perform mobile dexterous manipulation from human videos in a variety of environments. Watch Episode #45 of RoboPapers today, hosted by Michael Cho - Rbt/Acc and Chris Paxton!
1:01:08

Just collecting manipulation data isn’t enough for robots - they need to be able to move around in the world, which has a whole different set of challenges from pure manipulation. And bringing navigation and manipulation together in a single framework is even more challenging. Enter HERMES, from Zhecheng Yuan and Tianming Wei. This is a four-stage process in which human videos are used to set up an RL sim-to-real training pipeline in order to overcome differences between robot and human kinematics, and used together with a navigation foundation model to move around in a variety of environments. To learn more, join us as Zhecheng Yuan and Tianming Wei tell us about how they built their system to perform mobile dexterous manipulation from human videos in a variety of environments. Watch Episode #45 of RoboPapers today, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

15,045 görüntüleme • 7 ay önce

Mobile-TeleVision follows our previous idea on upper-body and lower-body separation for control: 1⃣We train RL for lower-body locomotion and standing; 2⃣We use IK and retargeting directly for upper-body for high-precision and smooth manipulation. We avoid using RL for upper-body motion tracking. This is because fundamentally it is hard to avoid errors with RL tracking, and sim2real gap brings another gap. At this point, we still find using IK is more practical and precise.
1:15

Mobile-TeleVision follows our previous idea on upper-body and lower-body separation for control: 1⃣We train RL for lower-body locomotion and standing; 2⃣We use IK and retargeting directly for upper-body for high-precision and smooth manipulation. We avoid using RL for upper-body motion tracking. This is because fundamentally it is hard to avoid errors with RL tracking, and sim2real gap brings another gap. At this point, we still find using IK is more practical and precise.

Xiaolong Wang

32,662 görüntüleme • 1 yıl önce

The future of robot butlers starts with mobile manipulation. We’re announcing the NeurIPS 2023 Open-Vocabulary Mobile Manipulation Challenge! - Full robot stack ✅ - Parallel sim and real evaluation ✅ - No robot required ✅👀
1:24

The future of robot butlers starts with mobile manipulation. We’re announcing the NeurIPS 2023 Open-Vocabulary Mobile Manipulation Challenge! - Full robot stack ✅ - Parallel sim and real evaluation ✅ - No robot required ✅👀

Chris Paxton

178,939 görüntüleme • 3 yıl önce

🤖Introducing TWIST: Teleoperated Whole-Body Imitation System. We develop a humanoid teleoperation system to enable coordinated, versatile, whole-body movements, using a single neural network. This is our first step toward general-purpose robots. 🌐
1:30

🤖Introducing TWIST: Teleoperated Whole-Body Imitation System. We develop a humanoid teleoperation system to enable coordinated, versatile, whole-body movements, using a single neural network. This is our first step toward general-purpose robots. 🌐

Yanjie Ze

79,931 görüntüleme • 1 yıl önce