🚨 New research from my team at Apple - real-time augmented reality robot feedback with just your hands + Vision Pro! Paper: Short thread below -

Data is the bottleneck for robot learning, and teleoperation is bottlenecked by physical hardware. Can we collect high-quality trajectory data without robots?

With ARMADA, we run a live robot simulation that follows pose targets given by the human hands at 30 Hz and overlays robot state info in the visual field. This allows us to collect high-quality data that’s directly replayable on physical hardware.

We conducted a user study with 15 participants under different feedback conditions; results suggest that robot feedback dramatically improves demo quality, improving trajectory replay success rates from an average of 1.3% to 71.1%.

We also instructed users to collect data while *imagining* the robot feedback after having used it for a while. Interestingly, this condition had an average replay rate of 33.3%, suggesting that humans can *learn* to provide more robot-compatible demos.

For more info, check out - the arxiv preprint: the Apple MLR webpage: Great to work with wonderful collaborators Nataliya Nechyporenko (co-lead), Christopher Webb, Mouli Sivapurapu, and Jian Zhang

Excited about the latest tech for your drone product? Our NDAA-compliant, US-made flight controllers are designed to accelerate your path to market and provide a solid platform for developing your autonomous software. Check them out! #Drones #UAV #UAS #Robotics #MadeInUSA

The man that can save the Vision Pro

very cool