Загрузка видео...

Не удалось загрузить видео

На главную

With the recent progress in large-scale multi-task robot training, how can we advance the real-world deployment of multi-task robot fleets? Introducing Sirius-Fleet✨, a multi-task interactive robot fleet learning framework with 𝗩𝗶𝘀𝘂𝗮𝗹 𝗪𝗼𝗿𝗹𝗱 𝗠𝗼𝗱𝗲𝗹𝘀! 🌍 #CoRL2024

28,027 просмотров • 1 год назад •via X (Twitter)

Комментарии: 10

Фото профиля Huihan Liu
Huihan Liu1 год назад

Our recipe: Stage 1: Pre-train a visual world model on diverse datasets to 𝙥𝙧𝙚𝙙𝙞𝙘𝙩 𝙛𝙪𝙩𝙪𝙧𝙚 𝙤𝙪𝙩𝙘𝙤𝙢𝙚𝙨 across many tasks. Stage 2: Deploy a multi-task policy on a robot fleet, with anomaly predictors monitoring the fleet deployment via the visual world model.

Фото профиля Huihan Liu
Huihan Liu1 год назад

The visual world model predicts future embeddings. It learns a latent space using image reconstruction and predicts future embeddings with cVAE. We train anomaly predictors on the frozen future embeddings, including failure and Out-of-Distribution (OOD) predictors.

Фото профиля Huihan Liu
Huihan Liu1 год назад

The anomaly predictors and the multi-task policy are fine-tuned over time. Over deployment, Sirius-Fleet improves on 1️⃣Combined Policy Performance for human-robot teaming, 2️⃣Autonomous Policy Performance for policy success, and 3️⃣Return of Human Effort for human use efficiency.

Фото профиля Huihan Liu
Huihan Liu1 год назад

Check out our paper and the project website for more info! 📃 🌐 A huge thank you to the team @YZ_Franklin, Vaarij Betala, Evan Zhang, James Liu, Crystal Ding, and @yukez ! @texas_robotics

Фото профиля Appy Pie
Appy Pie1 год назад

Impressive! Visual World Models could truly revolutionize multi-task robot fleets.

Фото профиля عرفان Erfan
عرفان Erfan1 год назад

Cool

Фото профиля Ing Gan
Ing Gan1 год назад

Cool work. Is the world model trained on the observations in the same environment (i.e. trained for open door / turn on microwave separately)?

Фото профиля Huihan Liu
Huihan Liu1 год назад

Thanks! The world model is trained a wide range of tasks, not specific to one particular environment.

Фото профиля Heeger
Heeger1 год назад

Cool work! When will you present this work?

Фото профиля Huihan Liu
Huihan Liu1 год назад

Thanks! It will be Thursday afternoon, Session 3 at CoRL :)

Похожие видео