Advancing dexterous manipulation through scalable visual sim-to-real transfer. We are excited to share our RSS paper, “ViserDex: Visual Sim-to-Real for Robust Dexterous In-hand Reorientation.” 🌐 Project page: 1/N 🧵

4/N This work highlights 3D Gaussian Splatting as a practical path toward scalable RGB-only dexterous manipulation. For more details: This work was led by Arjun Bhardwaj (@ThougthShot )

2/N ViserDex introduces a sim-to-real framework for RGB-based in-hand reorientation using 3D Gaussian Splatting. We perform domain randomization directly in the Gaussian representation space to generate photorealistic and diverse training data for robust object pose estimation.

3/N Combined with curriculum-based reinforcement learning and teacher–student distillation, ViserDex enables real-world reorientation of diverse objects on a multi-fingered robotic hand, even under challenging lighting conditions.

ahahahahaahaha

Gaussian splatting seems to really be making changes in the world of robotics. The better it gets the better robots will get

High-dexterity manipulation needs minimal latency. Eliminating loop jitter and CPU overhead is key for sim-to-real stability here.