Loading video...
Video Failed to Load
Controlling a robot in AR is one thing. A real working use case is another. Air hockey vs my Vector Robot on SPECS 2024 virtual puck, real goalie. This thing is playful, but imagine how far it may go. Tell me your ideas in a commets. Open source: if... show more
81,948 views • 2 months ago •via X (Twitter)
33 Comments

More details about our journey and a link to Git:

Shotour to the ML gurru @stspanho !

@specs This is really cool dude. Nice work.

@specs Thanks a lot!!!

@specs bro is living my childhood dream.

@specs Holy fuck why isn’t this on my child’s YouTube reels feed. Get every 10 year old asking for specs for Xmas. Hurry! 🚀

@specs Cool project and Nice color

@specs That’s pretty cool. Can you side load it quest?

@specs Hi! Yep the mac os side already have ability to play just through the Web UI, so you can build on top of it.

@specs This is fantastic, I am fascinated by AR, or mixed reality as Meta call it, I have an app that uses it heavily and earns me a modest living now.

@specs AR games with robots will be thing huh? Great work!

@specs this is so cool

@specs AR League of legends IRL would be the ultimate Laser Tag killer

@specs Amazing! 🔥

@specs bro made pong but the paddle has feelings and can be upset with you

@specs I have one of those gathering dust. I love this project

@specs what's your latency budget ar air hockey breaks once vector starts guessing

@specs we once swapped our Frame identity into a Vector during a test run. The puck deflection felt exactly like our own learned timing. ⚡🏒

@specs the server dependency is the real wall i hit building ar+hardware stuff losing the mac middleman means you can actually hand this to someone and it just works 🔥

@specs Your project inspired me to build an AR tower defense game with Meta Quest and Vector! Unfortunately, my hardware was different, so I had to retrain the robot detection model. Thanks for the inspiration!

@specs This is super cool and impressive! Great work. How did you go about designing/building this? Would be interested in kearn more

@specs Overall strategy was design from constraints that shaped entire experiance and idea: - limited dynamics of robot and drift on rotation -> shaped the motion - FVO of Specacles & limmited space on table ahead of you -> shaped the game i picked - Style was just a choise )

@specs this is so cool, imagine robots learning to play with us in real time!

@specs This is what xreal could have been smh

@specs 👀 oh baby that looks good

@specs specs costs $3500, too much

@specs virtual puck real goalie is such a clean proof of concept ondevice inference on the glasses tho… that changes everything

Spectacles running on-device spatial compute + real-time robot control — the "strong" vertex pushed to new territory. Open question: battery life running VectAR fully on glasses? The compute feat is clear. The thermal / power envelope decides if this graduates from demo to product.

@specs this is so cool, dude.

@specs this is the coolest vector integration i have ever seen. I might dust off my gen 1 vector

@specs Amazing

@specs Love this - going to dust off my vector now...

@specs I have one of these and the cubes lol imagine this connected with Hermes would be awesome

