Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Introducing Open-World Mobile Manipulation 🦾🌍 – A full-stack approach for operating articulated objects in open-ended unstructured environments: Unlocking doors with lever handles/ round knobs/ spring-loaded hinges 🔓🚪 Opening cabinets, drawers, and refrigerators 🗄️ 👇 Led by: @HaoyuXiong1 , Russell Mendonca , Kenny Shaw Deepak Pathak 1/n

Haoyu_Xiong_'s profile picture

Haoyu Xiong

3,692 subscribers

169,888 görüntüleme • 2 yıl önce •via X (Twitter)

Sağlık & İyilikEğitimBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

10 Yorum

Haoyu Xiong profil fotoğrafı
Haoyu Xiong2 yıl önce

Starting from a fixed set of demos on a fixed set of known objects, how do we deal with diverse unseen objects in the open world? Just keep adapting online! Our robot learns to operate new objects in under an hour via practice using RL, without requiring any new demos! 2/n

Haoyu Xiong profil fotoğrafı
Haoyu Xiong2 yıl önce

Realistic tasks like real-world door opening need good hardware: -- High payload for repeated interaction with real objects, e.g. a heavy, spring-loaded door, -- Agility to navigate around cluttered spaces. We provide low-cost hardware with around 25k USD 👇 3/n

Haoyu Xiong profil fotoğrafı
Haoyu Xiong2 yıl önce

We conducted a large field test around Carnegie Mellon Campus @SCSatCMU @CMU_Robotics to validate the effectiveness of our system. Our online adaptation framework improved the initial BC performance from 50% to 95%.

Ken Liu profil fotoğrafı
Ken Liu2 yıl önce

@HaoyuXiong1 Congrats on the cool work!!

Haoyu Xiong profil fotoğrafı
Haoyu Xiong2 yıl önce

Thank you @kenziyuliu

Ted Xiao profil fotoğrafı
Ted Xiao2 yıl önce

@HaoyuXiong1 Nice work!

Haoyu Xiong profil fotoğrafı
Haoyu Xiong2 yıl önce

thank you @xiao_ted

Simon Guo 🦝 profil fotoğrafı
Simon Guo 🦝2 yıl önce

@HaoyuXiong1 @kenny__shaw Kenny this is so cool!! Great job!!

Karol Hausman profil fotoğrafı
Karol Hausman2 yıl önce

@HaoyuXiong1 Cool work, congrats!

Haoyu Xiong profil fotoğrafı
Haoyu Xiong2 yıl önce

Thank you ! @hausman_k

Benzer Videolar

The future of robot butlers starts with mobile manipulation. We’re announcing the NeurIPS 2023 Open-Vocabulary Mobile Manipulation Challenge! - Full robot stack ✅ - Parallel sim and real evaluation ✅ - No robot required ✅👀
1:24

The future of robot butlers starts with mobile manipulation. We’re announcing the NeurIPS 2023 Open-Vocabulary Mobile Manipulation Challenge! - Full robot stack ✅ - Parallel sim and real evaluation ✅ - No robot required ✅👀

Chris Paxton

178,966 görüntüleme • 3 yıl önce

What representation enables open-world robot manipulation from generated videos? Introducing Dream2Flow, our recent work that bridges video generation and robot control with 3D object flow. Stanford University #ICRA2026 1/N
1:08

What representation enables open-world robot manipulation from generated videos? Introducing Dream2Flow, our recent work that bridges video generation and robot control with 3D object flow. Stanford University #ICRA2026 1/N

Wenlong Huang

105,597 görüntüleme • 3 ay önce

The doors are open 🚪 Iga Swiatek and Caty McNally take to Centre Court for their second round meeting 🤝 #Wimbledon
0:22

The doors are open 🚪 Iga Swiatek and Caty McNally take to Centre Court for their second round meeting 🤝 #Wimbledon

Wimbledon

51,666 görüntüleme • 1 yıl önce

Spatial reasoning is a major challenge for the foundation models today, even in simple tasks like arranging objects in 3D space. #CVPR2025 Introducing LayoutVLM, a differentiable optimization framework that uses VLM to spatially reason about diverse scene layouts from unlabeled assets and open-ended language instructions 1/n
0:45

Spatial reasoning is a major challenge for the foundation models today, even in simple tasks like arranging objects in 3D space. #CVPR2025 Introducing LayoutVLM, a differentiable optimization framework that uses VLM to spatially reason about diverse scene layouts from unlabeled assets and open-ended language instructions 1/n

Fan-Yun Sun

92,545 görüntüleme • 1 yıl önce

DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects abs: paper page:
0:27

DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects abs: paper page:

AK

17,554 görüntüleme • 3 yıl önce

I am thrilled to introduce OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code. Led by Maxence Faldor and Jenny Zhang, with Antoine Cully and myself. 🧵👇
1:03

I am thrilled to introduce OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code. Led by Maxence Faldor and Jenny Zhang, with Antoine Cully and myself. 🧵👇

Jeff Clune

170,209 görüntüleme • 2 yıl önce

🎤🎤 Excited to introduce COME-robot🤖🤖, Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V. It is the first closed-loop framework utilizing the vision-language foundation model for open-ended reasoning and adaptive planning in real-world scenarios. COME-robot demonstrates a significant improvement in task success rate (~25%) compared to SOTA methods. Project: Arxiv:
0:58

🎤🎤 Excited to introduce COME-robot🤖🤖, Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V. It is the first closed-loop framework utilizing the vision-language foundation model for open-ended reasoning and adaptive planning in real-world scenarios. COME-robot demonstrates a significant improvement in task success rate (~25%) compared to SOTA methods. Project: Arxiv:

Siyuan Huang

22,291 görüntüleme • 2 yıl önce

Money open doors which don't have door handles
0:41

Money open doors which don't have door handles

DJ Coach

77,082 görüntüleme • 1 yıl önce

For robots to be useful, they must be able to interact with a wide variety of environments; and yet, scaling interaction data is difficult, expensive, and time consuming. Instead, much research revolves around sim-to-real manipulation — but mostly this has not been mobile manipulation. Recently, though, this has begun to change. Two recent papers from Tairan He and Haoru Xue show us how to unlock the potential of this technique, building policies which, without any real data at all, can move objects around in the world and open doors in the real world with a humanoid robot. Watch Episode #60 of RoboPapers now to learn more, hosted by Chris Paxton and Jiafei Duan. In this episode, we cover two papers:. First is VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation; and second is DoorMan: Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer.
1:13:32

For robots to be useful, they must be able to interact with a wide variety of environments; and yet, scaling interaction data is difficult, expensive, and time consuming. Instead, much research revolves around sim-to-real manipulation — but mostly this has not been mobile manipulation. Recently, though, this has begun to change. Two recent papers from Tairan He and Haoru Xue show us how to unlock the potential of this technique, building policies which, without any real data at all, can move objects around in the world and open doors in the real world with a humanoid robot. Watch Episode #60 of RoboPapers now to learn more, hosted by Chris Paxton and Jiafei Duan. In this episode, we cover two papers:. First is VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation; and second is DoorMan: Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer.

RoboPapers

30,767 görüntüleme • 5 ay önce

Introducing: the first $10,000 of funding for open source contributions to the project, powered by the Merit Systems financial stack for open source and funded on Solana. Push a PR and start earning money today:
1:30

Introducing: the first $10,000 of funding for open source contributions to the project, powered by the Merit Systems financial stack for open source and funded on Solana. Push a PR and start earning money today:

Dark

100,451 görüntüleme • 8 ay önce

Introducing ♾OmniJARVIS, our latest venture to #AgentGPT, or vision-language-action (VLA) models for open-world instruction-following agents 🦾🕹️ tuning in 👉 by Team CraftJarvis, 🤿⏬
2:16

Introducing ♾OmniJARVIS, our latest venture to #AgentGPT, or vision-language-action (VLA) models for open-world instruction-following agents 🦾🕹️ tuning in 👉 by Team CraftJarvis, 🤿⏬

Xiaojian Ma

32,539 görüntüleme • 2 yıl önce

Introducing ManipGen: a sim2real agent for zero-shot manipulation. ManipGen handles complex tasks in the real world like organizing shelves, tidying cluttered tables, and more – all from text input and with on human demonstrations!
0:20

Introducing ManipGen: a sim2real agent for zero-shot manipulation. ManipGen handles complex tasks in the real world like organizing shelves, tidying cluttered tables, and more – all from text input and with on human demonstrations!

Deepak Pathak

57,936 görüntüleme • 1 yıl önce

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N
1:15

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N

Murtaza Dalal

114,538 görüntüleme • 1 yıl önce

Dream2Flow Bridging Video Generation and Open-World Manipulation with 3D Object Flow
2:59

Dream2Flow Bridging Video Generation and Open-World Manipulation with 3D Object Flow

AK

29,980 görüntüleme • 6 ay önce

INTRODUCING SLASH INCORPORATION 📃 Open a US LLC, C Corp, or S Corp in 10 minutes and get: ✅ An EIN and Operating Agreement ✅ 1 year of Registered Agent Services ✅ A virtual mailbox and business address All for $200 🤑 Open a business today:
0:28

INTRODUCING SLASH INCORPORATION 📃 Open a US LLC, C Corp, or S Corp in 10 minutes and get: ✅ An EIN and Operating Agreement ✅ 1 year of Registered Agent Services ✅ A virtual mailbox and business address All for $200 🤑 Open a business today:

Slash

38,845 görüntüleme • 2 yıl önce

3! 2! 1! Buc-ee's in Ohio is now officially open! A line wrapped around the store ahead of opening Monday morning, with doors opening at 6 a.m. What to know:
0:36

3! 2! 1! Buc-ee's in Ohio is now officially open! A line wrapped around the store ahead of opening Monday morning, with doors opening at 6 a.m. What to know:

WLWT

209,470 görüntüleme • 2 ay önce

introducing VibeMarketer skills an open-source skill package for agentic marketing workflows. works with Codex, Claude Code, and similar agent environments. demo + repo below.
12:39

introducing VibeMarketer skills an open-source skill package for agentic marketing workflows. works with Codex, Claude Code, and similar agent environments. demo + repo below.

J.B.

13,785 görüntüleme • 21 gün önce

CRAFT hand🫳 1. Achieves all 33/33 dexterous grasps > 2x-20x $$ hands! 2. < $600 3. Handles fragile objects 4. Durable under contact 5. Open-sourced Leo Lin & Shivansh Patel (on market; hire him🚀) will happily share anything else that you may need. Details in 🧵 🌟 Big shout out to Kenny Shaw (Leap & v2), Irmak Guzey (RUKA), ORCA Dexterity (ORCA), and many others who helped build this open research community. Thank you!
0:24

CRAFT hand🫳 1. Achieves all 33/33 dexterous grasps > 2x-20x $$ hands! 2. < $600 3. Handles fragile objects 4. Durable under contact 5. Open-sourced Leo Lin & Shivansh Patel (on market; hire him🚀) will happily share anything else that you may need. Details in 🧵 🌟 Big shout out to Kenny Shaw (Leap & v2), Irmak Guzey (RUKA), ORCA Dexterity (ORCA), and many others who helped build this open research community. Thank you!

Unnat Jain

35,443 görüntüleme • 2 ay önce

We have seen a lot of legged robots doing navigation in the wild. But how about mobile manipulation in the wild? I have been pushing the direction of learning a unified, efficient, and dynamic 3D representation of scenes (for navigation) and objects (for manipulation) for the past two years. And now we have GeFF --- our large-scale, generalizable feature field, that combines the speed of a feed-forward neural network with the rich semantics from Foundation Models, to handle dynamically changing scenes, and enable open-ended, language-grounded scene and object understanding.
1:45

We have seen a lot of legged robots doing navigation in the wild. But how about mobile manipulation in the wild? I have been pushing the direction of learning a unified, efficient, and dynamic 3D representation of scenes (for navigation) and objects (for manipulation) for the past two years. And now we have GeFF --- our large-scale, generalizable feature field, that combines the speed of a feed-forward neural network with the rich semantics from Foundation Models, to handle dynamically changing scenes, and enable open-ended, language-grounded scene and object understanding.

Xiaolong Wang

42,767 görüntüleme • 2 yıl önce

Five years ago, it started with a single upload. Today, our video series enters a new chapter. Introducing Access Granted, a rebrand of our documentary series, focused on opening doors and bringing viewers inside some of the most fascinating places in the world. Excited for what’s ahead. #AccessGranted 🔓
1:26

Five years ago, it started with a single upload. Today, our video series enters a new chapter. Introducing Access Granted, a rebrand of our documentary series, focused on opening doors and bringing viewers inside some of the most fascinating places in the world. Excited for what’s ahead. #AccessGranted 🔓

Sam Eckholm

20,998 görüntüleme • 5 ay önce