Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Introducing Open-World Mobile Manipulation 🦾🌍 – A full-stack approach for operating articulated objects in open-ended unstructured environments: Unlocking doors with lever handles/ round knobs/ spring-loaded hinges 🔓🚪 Opening cabinets, drawers, and refrigerators 🗄️ 👇 Led by: @HaoyuXiong1 , Russell Mendonca , Kenny Shaw Deepak Pathak 1/n

Haoyu_Xiong_'s profile picture

Haoyu Xiong

3,692 subscribers

169,888 Aufrufe • vor 2 Jahren •via X (Twitter)

Gesundheit & WellnessBildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Haoyu Xiong
Haoyu Xiongvor 2 Jahren

Starting from a fixed set of demos on a fixed set of known objects, how do we deal with diverse unseen objects in the open world? Just keep adapting online! Our robot learns to operate new objects in under an hour via practice using RL, without requiring any new demos! 2/n

Profilbild von Haoyu Xiong
Haoyu Xiongvor 2 Jahren

Realistic tasks like real-world door opening need good hardware: -- High payload for repeated interaction with real objects, e.g. a heavy, spring-loaded door, -- Agility to navigate around cluttered spaces. We provide low-cost hardware with around 25k USD 👇 3/n

Profilbild von Haoyu Xiong
Haoyu Xiongvor 2 Jahren

We conducted a large field test around Carnegie Mellon Campus @SCSatCMU @CMU_Robotics to validate the effectiveness of our system. Our online adaptation framework improved the initial BC performance from 50% to 95%.

Profilbild von Ken Liu
Ken Liuvor 2 Jahren

@HaoyuXiong1 Congrats on the cool work!!

Profilbild von Haoyu Xiong
Haoyu Xiongvor 2 Jahren

Thank you @kenziyuliu

Profilbild von Ted Xiao
Ted Xiaovor 2 Jahren

@HaoyuXiong1 Nice work!

Profilbild von Haoyu Xiong
Haoyu Xiongvor 2 Jahren

thank you @xiao_ted

Profilbild von Simon Guo 🦝
Simon Guo 🦝vor 2 Jahren

@HaoyuXiong1 @kenny__shaw Kenny this is so cool!! Great job!!

Profilbild von Karol Hausman
Karol Hausmanvor 2 Jahren

@HaoyuXiong1 Cool work, congrats!

Profilbild von Haoyu Xiong
Haoyu Xiongvor 2 Jahren

Thank you ! @hausman_k

Ähnliche Videos

The future of robot butlers starts with mobile manipulation. We’re announcing the NeurIPS 2023 Open-Vocabulary Mobile Manipulation Challenge! - Full robot stack ✅ - Parallel sim and real evaluation ✅ - No robot required ✅👀
1:24

The future of robot butlers starts with mobile manipulation. We’re announcing the NeurIPS 2023 Open-Vocabulary Mobile Manipulation Challenge! - Full robot stack ✅ - Parallel sim and real evaluation ✅ - No robot required ✅👀

Chris Paxton

178,939 Aufrufe • vor 3 Jahren

What representation enables open-world robot manipulation from generated videos? Introducing Dream2Flow, our recent work that bridges video generation and robot control with 3D object flow. Stanford University #ICRA2026 1/N
1:08

What representation enables open-world robot manipulation from generated videos? Introducing Dream2Flow, our recent work that bridges video generation and robot control with 3D object flow. Stanford University #ICRA2026 1/N

Wenlong Huang

105,597 Aufrufe • vor 3 Monaten

The doors are open 🚪 Iga Swiatek and Caty McNally take to Centre Court for their second round meeting 🤝 #Wimbledon
0:22

The doors are open 🚪 Iga Swiatek and Caty McNally take to Centre Court for their second round meeting 🤝 #Wimbledon

Wimbledon

51,666 Aufrufe • vor 1 Jahr

Spatial reasoning is a major challenge for the foundation models today, even in simple tasks like arranging objects in 3D space. #CVPR2025 Introducing LayoutVLM, a differentiable optimization framework that uses VLM to spatially reason about diverse scene layouts from unlabeled assets and open-ended language instructions 1/n
0:45

Spatial reasoning is a major challenge for the foundation models today, even in simple tasks like arranging objects in 3D space. #CVPR2025 Introducing LayoutVLM, a differentiable optimization framework that uses VLM to spatially reason about diverse scene layouts from unlabeled assets and open-ended language instructions 1/n

Fan-Yun Sun

92,545 Aufrufe • vor 1 Jahr

DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects abs: paper page:
0:27

DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects abs: paper page:

AK

17,554 Aufrufe • vor 3 Jahren

I am thrilled to introduce OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code. Led by Maxence Faldor and Jenny Zhang, with Antoine Cully and myself. 🧵👇
1:03

I am thrilled to introduce OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code. Led by Maxence Faldor and Jenny Zhang, with Antoine Cully and myself. 🧵👇

Jeff Clune

170,209 Aufrufe • vor 2 Jahren

🎤🎤 Excited to introduce COME-robot🤖🤖, Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V. It is the first closed-loop framework utilizing the vision-language foundation model for open-ended reasoning and adaptive planning in real-world scenarios. COME-robot demonstrates a significant improvement in task success rate (~25%) compared to SOTA methods. Project: Arxiv:
0:58

🎤🎤 Excited to introduce COME-robot🤖🤖, Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V. It is the first closed-loop framework utilizing the vision-language foundation model for open-ended reasoning and adaptive planning in real-world scenarios. COME-robot demonstrates a significant improvement in task success rate (~25%) compared to SOTA methods. Project: Arxiv:

Siyuan Huang

22,291 Aufrufe • vor 2 Jahren

Money open doors which don't have door handles
0:41

Money open doors which don't have door handles

DJ Coach

77,082 Aufrufe • vor 1 Jahr

For robots to be useful, they must be able to interact with a wide variety of environments; and yet, scaling interaction data is difficult, expensive, and time consuming. Instead, much research revolves around sim-to-real manipulation — but mostly this has not been mobile manipulation. Recently, though, this has begun to change. Two recent papers from Tairan He and Haoru Xue show us how to unlock the potential of this technique, building policies which, without any real data at all, can move objects around in the world and open doors in the real world with a humanoid robot. Watch Episode #60 of RoboPapers now to learn more, hosted by Chris Paxton and Jiafei Duan. In this episode, we cover two papers:. First is VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation; and second is DoorMan: Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer.
1:13:32

For robots to be useful, they must be able to interact with a wide variety of environments; and yet, scaling interaction data is difficult, expensive, and time consuming. Instead, much research revolves around sim-to-real manipulation — but mostly this has not been mobile manipulation. Recently, though, this has begun to change. Two recent papers from Tairan He and Haoru Xue show us how to unlock the potential of this technique, building policies which, without any real data at all, can move objects around in the world and open doors in the real world with a humanoid robot. Watch Episode #60 of RoboPapers now to learn more, hosted by Chris Paxton and Jiafei Duan. In this episode, we cover two papers:. First is VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation; and second is DoorMan: Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer.

RoboPapers

30,767 Aufrufe • vor 5 Monaten

Introducing: the first $10,000 of funding for open source contributions to the project, powered by the Merit Systems financial stack for open source and funded on Solana. Push a PR and start earning money today:
1:30

Introducing: the first $10,000 of funding for open source contributions to the project, powered by the Merit Systems financial stack for open source and funded on Solana. Push a PR and start earning money today:

Dark

100,451 Aufrufe • vor 8 Monaten

Introducing ♾OmniJARVIS, our latest venture to #AgentGPT, or vision-language-action (VLA) models for open-world instruction-following agents 🦾🕹️ tuning in 👉 by Team CraftJarvis, 🤿⏬
2:16

Introducing ♾OmniJARVIS, our latest venture to #AgentGPT, or vision-language-action (VLA) models for open-world instruction-following agents 🦾🕹️ tuning in 👉 by Team CraftJarvis, 🤿⏬

Xiaojian Ma

32,539 Aufrufe • vor 2 Jahren

Introducing ManipGen: a sim2real agent for zero-shot manipulation. ManipGen handles complex tasks in the real world like organizing shelves, tidying cluttered tables, and more – all from text input and with on human demonstrations!
0:20

Introducing ManipGen: a sim2real agent for zero-shot manipulation. ManipGen handles complex tasks in the real world like organizing shelves, tidying cluttered tables, and more – all from text input and with on human demonstrations!

Deepak Pathak

57,936 Aufrufe • vor 1 Jahr

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N
1:15

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N

Murtaza Dalal

114,538 Aufrufe • vor 1 Jahr

Dream2Flow Bridging Video Generation and Open-World Manipulation with 3D Object Flow
2:59

Dream2Flow Bridging Video Generation and Open-World Manipulation with 3D Object Flow

AK

29,980 Aufrufe • vor 6 Monaten

INTRODUCING SLASH INCORPORATION 📃 Open a US LLC, C Corp, or S Corp in 10 minutes and get: ✅ An EIN and Operating Agreement ✅ 1 year of Registered Agent Services ✅ A virtual mailbox and business address All for $200 🤑 Open a business today:
0:28

INTRODUCING SLASH INCORPORATION 📃 Open a US LLC, C Corp, or S Corp in 10 minutes and get: ✅ An EIN and Operating Agreement ✅ 1 year of Registered Agent Services ✅ A virtual mailbox and business address All for $200 🤑 Open a business today:

Slash

38,845 Aufrufe • vor 2 Jahren

3! 2! 1! Buc-ee's in Ohio is now officially open! A line wrapped around the store ahead of opening Monday morning, with doors opening at 6 a.m. What to know:
0:36

3! 2! 1! Buc-ee's in Ohio is now officially open! A line wrapped around the store ahead of opening Monday morning, with doors opening at 6 a.m. What to know:

WLWT

209,470 Aufrufe • vor 2 Monaten

introducing VibeMarketer skills an open-source skill package for agentic marketing workflows. works with Codex, Claude Code, and similar agent environments. demo + repo below.
12:39

introducing VibeMarketer skills an open-source skill package for agentic marketing workflows. works with Codex, Claude Code, and similar agent environments. demo + repo below.

J.B.

13,785 Aufrufe • vor 21 Tagen

CRAFT hand🫳 1. Achieves all 33/33 dexterous grasps > 2x-20x $$ hands! 2. < $600 3. Handles fragile objects 4. Durable under contact 5. Open-sourced Leo Lin & Shivansh Patel (on market; hire him🚀) will happily share anything else that you may need. Details in 🧵 🌟 Big shout out to Kenny Shaw (Leap & v2), Irmak Guzey (RUKA), ORCA Dexterity (ORCA), and many others who helped build this open research community. Thank you!
0:24

CRAFT hand🫳 1. Achieves all 33/33 dexterous grasps > 2x-20x $$ hands! 2. < $600 3. Handles fragile objects 4. Durable under contact 5. Open-sourced Leo Lin & Shivansh Patel (on market; hire him🚀) will happily share anything else that you may need. Details in 🧵 🌟 Big shout out to Kenny Shaw (Leap & v2), Irmak Guzey (RUKA), ORCA Dexterity (ORCA), and many others who helped build this open research community. Thank you!

Unnat Jain

35,443 Aufrufe • vor 2 Monaten

We have seen a lot of legged robots doing navigation in the wild. But how about mobile manipulation in the wild? I have been pushing the direction of learning a unified, efficient, and dynamic 3D representation of scenes (for navigation) and objects (for manipulation) for the past two years. And now we have GeFF --- our large-scale, generalizable feature field, that combines the speed of a feed-forward neural network with the rich semantics from Foundation Models, to handle dynamically changing scenes, and enable open-ended, language-grounded scene and object understanding.
1:45

We have seen a lot of legged robots doing navigation in the wild. But how about mobile manipulation in the wild? I have been pushing the direction of learning a unified, efficient, and dynamic 3D representation of scenes (for navigation) and objects (for manipulation) for the past two years. And now we have GeFF --- our large-scale, generalizable feature field, that combines the speed of a feed-forward neural network with the rich semantics from Foundation Models, to handle dynamically changing scenes, and enable open-ended, language-grounded scene and object understanding.

Xiaolong Wang

42,767 Aufrufe • vor 2 Jahren

Five years ago, it started with a single upload. Today, our video series enters a new chapter. Introducing Access Granted, a rebrand of our documentary series, focused on opening doors and bringing viewers inside some of the most fascinating places in the world. Excited for what’s ahead. #AccessGranted 🔓
1:26

Five years ago, it started with a single upload. Today, our video series enters a new chapter. Introducing Access Granted, a rebrand of our documentary series, focused on opening doors and bringing viewers inside some of the most fascinating places in the world. Excited for what’s ahead. #AccessGranted 🔓

Sam Eckholm

20,998 Aufrufe • vor 5 Monaten