Introducing Open-World Mobile Manipulation 🦾🌍 – A full-stack approach for operating articulated objects in open-ended unstructured environments: Unlocking doors with lever handles/ round knobs/ spring-loaded hinges 🔓🚪 Opening cabinets, drawers, and refrigerators 🗄️ 👇 Led by: @HaoyuXiong1 , Russell Mendonca , Kenny Shaw Deepak Pathak 1/n

Starting from a fixed set of demos on a fixed set of known objects, how do we deal with diverse unseen objects in the open world? Just keep adapting online! Our robot learns to operate new objects in under an hour via practice using RL, without requiring any new demos! 2/n

Realistic tasks like real-world door opening need good hardware: -- High payload for repeated interaction with real objects, e.g. a heavy, spring-loaded door, -- Agility to navigate around cluttered spaces. We provide low-cost hardware with around 25k USD 👇 3/n

We conducted a large field test around Carnegie Mellon Campus @SCSatCMU @CMU_Robotics to validate the effectiveness of our system. Our online adaptation framework improved the initial BC performance from 50% to 95%.

@HaoyuXiong1 Congrats on the cool work!!

Thank you @kenziyuliu

@HaoyuXiong1 Nice work!

thank you @xiao_ted

@HaoyuXiong1 @kenny__shaw Kenny this is so cool!! Great job!!

@HaoyuXiong1 Cool work, congrats!

Thank you ! @hausman_k