正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Introducing Open-World Mobile Manipulation 🦾🌍 – A full-stack approach for operating articulated objects in open-ended unstructured environments: Unlocking doors with lever handles/ round knobs/ spring-loaded hinges 🔓🚪 Opening cabinets, drawers, and refrigerators 🗄️ 👇 Led by: @HaoyuXiong1 , Russell Mendonca , Kenny Shaw Deepak Pathak 1/n

Haoyu Xiong

3,692 subscribers

169,888 次观看 • 2 年前 •via X (Twitter)

健康养生教育科学技术

Anya Rossi• Live Now

Private livecam show

10 条评论

Haoyu Xiong 的头像

Haoyu Xiong2 年前

Starting from a fixed set of demos on a fixed set of known objects, how do we deal with diverse unseen objects in the open world? Just keep adapting online! Our robot learns to operate new objects in under an hour via practice using RL, without requiring any new demos! 2/n

Haoyu Xiong 的头像

Haoyu Xiong2 年前

Realistic tasks like real-world door opening need good hardware: -- High payload for repeated interaction with real objects, e.g. a heavy, spring-loaded door, -- Agility to navigate around cluttered spaces. We provide low-cost hardware with around 25k USD 👇 3/n

Haoyu Xiong 的头像

Haoyu Xiong2 年前

We conducted a large field test around Carnegie Mellon Campus @SCSatCMU @CMU_Robotics to validate the effectiveness of our system. Our online adaptation framework improved the initial BC performance from 50% to 95%.

Ken Liu 的头像

Ken Liu2 年前

@HaoyuXiong1 Congrats on the cool work!!

Haoyu Xiong 的头像

Haoyu Xiong2 年前

Thank you @kenziyuliu

Ted Xiao 的头像

Ted Xiao2 年前

@HaoyuXiong1 Nice work!

Haoyu Xiong 的头像

Haoyu Xiong2 年前

thank you @xiao_ted

Simon Guo 🦝 的头像

Simon Guo 🦝2 年前

@HaoyuXiong1 @kenny__shaw Kenny this is so cool!! Great job!!

Karol Hausman 的头像

Karol Hausman2 年前

@HaoyuXiong1 Cool work, congrats!

Haoyu Xiong 的头像

Haoyu Xiong2 年前

Thank you ! @hausman_k

相关视频

The future of robot butlers starts with mobile manipulation. We’re announcing the NeurIPS 2023 Open-Vocabulary Mobile Manipulation Challenge! - Full robot stack ✅ - Parallel sim and real evaluation ✅ - No robot required ✅👀

The future of robot butlers starts with mobile manipulation. We’re announcing the NeurIPS 2023 Open-Vocabulary Mobile Manipulation Challenge! - Full robot stack ✅ - Parallel sim and real evaluation ✅ - No robot required ✅👀

Chris Paxton

178,966 次观看 • 3 年前

What representation enables open-world robot manipulation from generated videos? Introducing Dream2Flow, our recent work that bridges video generation and robot control with 3D object flow. Stanford University #ICRA2026 1/N

What representation enables open-world robot manipulation from generated videos? Introducing Dream2Flow, our recent work that bridges video generation and robot control with 3D object flow. Stanford University #ICRA2026 1/N

Wenlong Huang

105,597 次观看 • 3 个月前

The doors are open 🚪 Iga Swiatek and Caty McNally take to Centre Court for their second round meeting 🤝 #Wimbledon

The doors are open 🚪 Iga Swiatek and Caty McNally take to Centre Court for their second round meeting 🤝 #Wimbledon

Wimbledon

51,666 次观看 • 1 年前

Spatial reasoning is a major challenge for the foundation models today, even in simple tasks like arranging objects in 3D space. #CVPR2025 Introducing LayoutVLM, a differentiable optimization framework that uses VLM to spatially reason about diverse scene layouts from unlabeled assets and open-ended language instructions 1/n

Spatial reasoning is a major challenge for the foundation models today, even in simple tasks like arranging objects in 3D space. #CVPR2025 Introducing LayoutVLM, a differentiable optimization framework that uses VLM to spatially reason about diverse scene layouts from unlabeled assets and open-ended language instructions 1/n

Fan-Yun Sun

92,545 次观看 • 1 年前

DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects abs: paper page:

DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects abs: paper page:

AK

17,554 次观看 • 3 年前

I am thrilled to introduce OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code. Led by Maxence Faldor and Jenny Zhang, with Antoine Cully and myself. 🧵👇

I am thrilled to introduce OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code. Led by Maxence Faldor and Jenny Zhang, with Antoine Cully and myself. 🧵👇

Jeff Clune

170,209 次观看 • 2 年前

🎤🎤 Excited to introduce COME-robot🤖🤖, Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V. It is the first closed-loop framework utilizing the vision-language foundation model for open-ended reasoning and adaptive planning in real-world scenarios. COME-robot demonstrates a significant improvement in task success rate (~25%) compared to SOTA methods. Project: Arxiv:

🎤🎤 Excited to introduce COME-robot🤖🤖, Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V. It is the first closed-loop framework utilizing the vision-language foundation model for open-ended reasoning and adaptive planning in real-world scenarios. COME-robot demonstrates a significant improvement in task success rate (~25%) compared to SOTA methods. Project: Arxiv:

Siyuan Huang

22,291 次观看 • 2 年前

Money open doors which don't have door handles

Money open doors which don't have door handles

DJ Coach

77,082 次观看 • 1 年前

For robots to be useful, they must be able to interact with a wide variety of environments; and yet, scaling interaction data is difficult, expensive, and time consuming. Instead, much research revolves around sim-to-real manipulation — but mostly this has not been mobile manipulation. Recently, though, this has begun to change. Two recent papers from Tairan He and Haoru Xue show us how to unlock the potential of this technique, building policies which, without any real data at all, can move objects around in the world and open doors in the real world with a humanoid robot. Watch Episode #60 of RoboPapers now to learn more, hosted by Chris Paxton and Jiafei Duan. In this episode, we cover two papers:. First is VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation; and second is DoorMan: Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer.

For robots to be useful, they must be able to interact with a wide variety of environments; and yet, scaling interaction data is difficult, expensive, and time consuming. Instead, much research revolves around sim-to-real manipulation — but mostly this has not been mobile manipulation. Recently, though, this has begun to change. Two recent papers from Tairan He and Haoru Xue show us how to unlock the potential of this technique, building policies which, without any real data at all, can move objects around in the world and open doors in the real world with a humanoid robot. Watch Episode #60 of RoboPapers now to learn more, hosted by Chris Paxton and Jiafei Duan. In this episode, we cover two papers:. First is VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation; and second is DoorMan: Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer.

RoboPapers

30,767 次观看 • 5 个月前

Introducing: the first $10,000 of funding for open source contributions to the project, powered by the Merit Systems financial stack for open source and funded on Solana. Push a PR and start earning money today:

Introducing: the first $10,000 of funding for open source contributions to the project, powered by the Merit Systems financial stack for open source and funded on Solana. Push a PR and start earning money today:

Dark

100,451 次观看 • 8 个月前

Introducing ♾OmniJARVIS, our latest venture to #AgentGPT, or vision-language-action (VLA) models for open-world instruction-following agents 🦾🕹️ tuning in 👉 by Team CraftJarvis, 🤿⏬

Introducing ♾OmniJARVIS, our latest venture to #AgentGPT, or vision-language-action (VLA) models for open-world instruction-following agents 🦾🕹️ tuning in 👉 by Team CraftJarvis, 🤿⏬

Xiaojian Ma

32,539 次观看 • 2 年前

Introducing ManipGen: a sim2real agent for zero-shot manipulation. ManipGen handles complex tasks in the real world like organizing shelves, tidying cluttered tables, and more – all from text input and with on human demonstrations!

Introducing ManipGen: a sim2real agent for zero-shot manipulation. ManipGen handles complex tasks in the real world like organizing shelves, tidying cluttered tables, and more – all from text input and with on human demonstrations!

Deepak Pathak

57,936 次观看 • 1 年前

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N

Murtaza Dalal

114,538 次观看 • 1 年前

Dream2Flow Bridging Video Generation and Open-World Manipulation with 3D Object Flow

Dream2Flow Bridging Video Generation and Open-World Manipulation with 3D Object Flow

AK

29,980 次观看 • 6 个月前

INTRODUCING SLASH INCORPORATION 📃 Open a US LLC, C Corp, or S Corp in 10 minutes and get: ✅ An EIN and Operating Agreement ✅ 1 year of Registered Agent Services ✅ A virtual mailbox and business address All for $200 🤑 Open a business today:

INTRODUCING SLASH INCORPORATION 📃 Open a US LLC, C Corp, or S Corp in 10 minutes and get: ✅ An EIN and Operating Agreement ✅ 1 year of Registered Agent Services ✅ A virtual mailbox and business address All for $200 🤑 Open a business today:

Slash

38,845 次观看 • 2 年前

3! 2! 1! Buc-ee's in Ohio is now officially open! A line wrapped around the store ahead of opening Monday morning, with doors opening at 6 a.m. What to know:

3! 2! 1! Buc-ee's in Ohio is now officially open! A line wrapped around the store ahead of opening Monday morning, with doors opening at 6 a.m. What to know:

WLWT

209,470 次观看 • 2 个月前

introducing VibeMarketer skills an open-source skill package for agentic marketing workflows. works with Codex, Claude Code, and similar agent environments. demo + repo below.

introducing VibeMarketer skills an open-source skill package for agentic marketing workflows. works with Codex, Claude Code, and similar agent environments. demo + repo below.

J.B.

13,785 次观看 • 21 天前

CRAFT hand🫳 1. Achieves all 33/33 dexterous grasps > 2x-20x $$ hands! 2. < $600 3. Handles fragile objects 4. Durable under contact 5. Open-sourced Leo Lin & Shivansh Patel (on market; hire him🚀) will happily share anything else that you may need. Details in 🧵 🌟 Big shout out to Kenny Shaw (Leap & v2), Irmak Guzey (RUKA), ORCA Dexterity (ORCA), and many others who helped build this open research community. Thank you!

CRAFT hand🫳 1. Achieves all 33/33 dexterous grasps > 2x-20x $$ hands! 2. < $600 3. Handles fragile objects 4. Durable under contact 5. Open-sourced Leo Lin & Shivansh Patel (on market; hire him🚀) will happily share anything else that you may need. Details in 🧵 🌟 Big shout out to Kenny Shaw (Leap & v2), Irmak Guzey (RUKA), ORCA Dexterity (ORCA), and many others who helped build this open research community. Thank you!

Unnat Jain

35,443 次观看 • 2 个月前

We have seen a lot of legged robots doing navigation in the wild. But how about mobile manipulation in the wild? I have been pushing the direction of learning a unified, efficient, and dynamic 3D representation of scenes (for navigation) and objects (for manipulation) for the past two years. And now we have GeFF --- our large-scale, generalizable feature field, that combines the speed of a feed-forward neural network with the rich semantics from Foundation Models, to handle dynamically changing scenes, and enable open-ended, language-grounded scene and object understanding.

We have seen a lot of legged robots doing navigation in the wild. But how about mobile manipulation in the wild? I have been pushing the direction of learning a unified, efficient, and dynamic 3D representation of scenes (for navigation) and objects (for manipulation) for the past two years. And now we have GeFF --- our large-scale, generalizable feature field, that combines the speed of a feed-forward neural network with the rich semantics from Foundation Models, to handle dynamically changing scenes, and enable open-ended, language-grounded scene and object understanding.

Xiaolong Wang

42,767 次观看 • 2 年前

Five years ago, it started with a single upload. Today, our video series enters a new chapter. Introducing Access Granted, a rebrand of our documentary series, focused on opening doors and bringing viewers inside some of the most fascinating places in the world. Excited for what’s ahead. #AccessGranted 🔓

Five years ago, it started with a single upload. Today, our video series enters a new chapter. Introducing Access Granted, a rebrand of our documentary series, focused on opening doors and bringing viewers inside some of the most fascinating places in the world. Excited for what’s ahead. #AccessGranted 🔓

Sam Eckholm

20,998 次观看 • 5 个月前