Introduce Open-𝐓𝐞𝐥𝐞𝐕𝐢𝐬𝐢𝐨𝐧🤖: ⁣ We need an intuitive and remote teleoperation interface to collect more robot data. 𝐓𝐞𝐥𝐞𝐕𝐢𝐬𝐢𝐨𝐧 lets you immersively operate a robot even if you are 3000 miles away, like in the movie 𝘈𝘷𝘢𝘵𝘢𝘳. Open-sourced!

Real-time stereo video streaming provides spatial/depth understanding, so the operator is confident about the objects’ locations. This enables fine manipulation with challenging objects such as transparent boxes.

An active neck plus IK and retargeting enables intuitive perception and actuation for the operator. The operator just needs to look at the points of interest intuitively and the robot will follow the same head movements.

The system is easily accessible from any device with a web browser (Vision Pro, Quest, even mac, iPad, iPhone…).

Not possible without the joint efforts of @Jialong_LI_UIM @AaronYANG2000 @EpisodeYang @xiaolonw Try now even if you don’t have a VR device! More videos, code, hardware, and dataset at:

This is useful！

Thanks Huazhe!

This is awesome, congrats Xuxin!

Thanks Quanting!

The experiments show that the proposed system outperforms baseline models in terms of task success rates and completion times. The use of stereo video input is found to be crucial for the operator's spatial understanding and task performance. full paper:

@vateseif @arbwes