正在加载视频...

视频加载失败

Imitation learning works™ – but you need good data 🥹 How to get high-quality visuotactile demos from a bimanual robot with multifingered hands, and learn smooth policies? Check our new work “Learning Visuotactile Skills with Two Multifingered Hands”! 🙌

61,504 次观看 • 2 年前 •via X (Twitter)

10 条评论

Toru 的头像
Toru2 年前

We tackle two key challenges: 1. Lack of affordable and accessible teleoperation systems for bimanual multifingered hands 2. Lack of good hand hardware equipped with touch sensing

Toru 的头像
Toru2 年前

To address the first challenge, we develop HATO ("dove" 🕊️ in Japanese), a low-cost Hands-Arms TeleOperation system using Meta Quest 2. Our system demonstrates precise motion control capabilities and human-like dexterity -- we even controlled our robot to play Hollow Knight!

Toru 的头像
Toru2 年前

To tackle the latter challenge, we adapt two @PSYONICinc prosthetic hands with touch sensors for research, enabling visuotactile data collection with HATO. We learn skills that can complete long-horizon, high-precision tasks and generalize to different environment settings.

Toru 的头像
Toru2 年前

We empirically investigate the effects of dataset size, sensing modality, and visual input preprocessing on policy learning. Results and analysis can be found in our paper:

Toru 的头像
Toru2 年前

We have also released a comprehensive software suite that supports efficient data collection, multimodal data processing, scalable policy learning, and smooth policy deployment. See our Github repo for more details:

Toru 的头像
Toru2 年前

Project page: Huge thanks to amazing collaborators -- @YZ_Franklin, @qiyang_li , @HaozhiQ , @brenthyi, @svlevine, and @JitendraMalikCV !!! from @berkeley_ai 🤖

Jitendra MALIK 的头像
Jitendra MALIK2 年前

There have been impressive recent results using tele-op trajectories for training robot policies. These are typically for hands that are parallel jaw grippers; work led by @ToruO_O in our lab at BAIR shows that one can do this now for multifingered hands with vision and touch.

Hongyu Li 的头像
Hongyu Li2 年前

Great work! Congrats

Haotian (Sean) Liu 的头像
Haotian (Sean) Liu2 年前

Wow, the Hollow Knight demo is superb! がんばって

tommy 的头像
tommy2 年前

awesome 🤩

相关视频