Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

With camera access on Apple Vision Pro, the possibilities are endless! In this POC, I combined Gemini 2.0 Flash with XR to create a realtime hands-free experience. 💡 It sees what you see, responds to your voice, and delivers extra info in XR. Below are some simple examples.

Stijn Spanhove

2,880 subscribers

13,858 views • 1 year ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

11 Comments

Stijn Spanhove1 year ago

Imagine fine-tuning it for your workflows to show manuals, business insights, or process guides. All hands-free with Gemini 2.0 realtime @GoogleARVR

Stijn Spanhove1 year ago

Solving math problems

Stijn Spanhove1 year ago

Translations

Stijn Spanhove1 year ago

Playing games

WillScuderi1 year ago

Great rundown of our new product using it on the road 🏎️ with your Mac and PC and then on planes ✈️ with your iPhone for watching movies is a game changer ✅ Aluminium Build ✅ Works with one cable for Mac, PC, iPhone (15 and later) ✅ 1080p/16m colours/300nits ✅ 0.2"/5mm thin

Bryan Keller1 year ago

Is this using an enterprise dev entitlement?

Stijn Spanhove1 year ago

Yes I'm using an enterprise API for camera access

Carlos Aguilar1 year ago

Is this using enterprise APIs?

Stijn Spanhove1 year ago

Yes with the enterprise API you can have camera access.

Dre Smith1 year ago

This is cool… I need to check out Gemini 2.0

Stijn Spanhove1 year ago

You should! 😉

Related Videos

I loved being one of the first directors to use Apple Vision Pro! I was completely blown away with how it helps you play, create, and explore new worlds. Watch Wicked now on the Apple TV app, and experience it in 3D on Apple Vision Pro.

I loved being one of the first directors to use Apple Vision Pro! I was completely blown away with how it helps you play, create, and explore new worlds. Watch Wicked now on the Apple TV app, and experience it in 3D on Apple Vision Pro.

Jon M. Chu

378,694 views • 1 year ago

Disney Imagineers cooked up a HoloTile floor, so you can walk around in virtual spaces and it adapts to your movements. In 5 years, this floor combined with Apple Vision Pro combined with GTA VII will be the most insane gaming experience ever.

Disney Imagineers cooked up a HoloTile floor, so you can walk around in virtual spaces and it adapts to your movements. In 5 years, this floor combined with Apple Vision Pro combined with GTA VII will be the most insane gaming experience ever.

Trung Phan

1,395,782 views • 2 years ago

AI + AR is going to change everything! I turned my photo into a 3D bobblehead in under 5 minutes using ChatGPT and Tripo3D. Dropped it into my space with Vision Pro. It’s wild. No wonder Apple and Meta are shifting some of their XR people onto AI teams.

AI + AR is going to change everything! I turned my photo into a 3D bobblehead in under 5 minutes using ChatGPT and Tripo3D. Dropped it into my space with Vision Pro. It’s wild. No wonder Apple and Meta are shifting some of their XR people onto AI teams.

Justin Ryan ᯅ

12,232 views • 1 year ago

With Copilot and agents, the possibilities are endless — we can’t wait to discover what you create.

With Copilot and agents, the possibilities are endless — we can’t wait to discover what you create.

Microsoft

73,303 views • 1 year ago

Introducing Gemini 2.0 Flash with Google AI Studio in PyCharm... Totally Free! 🤩 In just one minute, I'll show you how to connect your Google AI Studio account and start using the Gemini 2.0 Flash and Gemini 2.0 Flash Thinking models for free. Save this video to stay updated on all the latest Google DeepMind models available in JetBrains editors! PyCharm, a JetBrains IDE CodeGPT Google for Developers

Introducing Gemini 2.0 Flash with Google AI Studio in PyCharm... Totally Free! 🤩 In just one minute, I'll show you how to connect your Google AI Studio account and start using the Gemini 2.0 Flash and Gemini 2.0 Flash Thinking models for free. Save this video to stay updated on all the latest Google DeepMind models available in JetBrains editors! PyCharm, a JetBrains IDE CodeGPT Google for Developers

Daniel San

65,294 views • 1 year ago

Passthrough Camera access (PCA) on Quest may have taken a while and I already see some devs doubting the impact this will have for XR. I think the most fun thing about PCA is, that there are such smart devs out there, that we cannot even begin to imagine what some people will come up with, and I am very much looking forward to that. Below is another sample I made and included in my QuestCameraKit GitHub repo. It is using API calls to OpenAI vision and whisper and is capable of understanding and responding in virtually any language (English and German below):

Passthrough Camera access (PCA) on Quest may have taken a while and I already see some devs doubting the impact this will have for XR. I think the most fun thing about PCA is, that there are such smart devs out there, that we cannot even begin to imagine what some people will come up with, and I am very much looking forward to that. Below is another sample I made and included in my QuestCameraKit GitHub repo. It is using API calls to OpenAI vision and whisper and is capable of understanding and responding in virtually any language (English and German below):

Robi 🔜 Meta Connect ᯅ

11,831 views • 1 year ago

XREAL and Google are extending our strategic partnership in a new multi‑year agreement, with XREAL named a lead hardware partner for the Android XR ecosystem. Building on Project Aura (coming in 2026), we’re aligning our long‑term hardware roadmap with Android XR—bringing it to optical see‑through devices, including wired XR glasses, and expanding support for Android XR development. Read more: Also—peep the Android XR alien lighting up the Vegas Sphere at CES 2026 😎 #AndroidXR #XR #AR #XREAL #Google

XREAL and Google are extending our strategic partnership in a new multi‑year agreement, with XREAL named a lead hardware partner for the Android XR ecosystem. Building on Project Aura (coming in 2026), we’re aligning our long‑term hardware roadmap with Android XR—bringing it to optical see‑through devices, including wired XR glasses, and expanding support for Android XR development. Read more: Also—peep the Android XR alien lighting up the Vegas Sphere at CES 2026 😎 #AndroidXR #XR #AR #XREAL #Google

XREAL 👓

22,199 views • 6 months ago

Want to see how I made my 2 Day XR Game using Vision Pro, Unity and xAI ? Here's how I made it:

Want to see how I made my 2 Day XR Game using Vision Pro, Unity and xAI ? Here's how I made it:

Preston Platt

16,148 views • 1 year ago

Imagine stepping into your embeddings that power RAG. 📚 In this proof of concept, you’re surrounded by courses. Semantically similar ones float right next to each other. I've built an XR prototype with Rémi at In The Pocket where you can stand inside your knowledge base on Apple Vision Pro. Ask a question and the AI lights up the relevant documents in space, so you literally see where your answer comes from. When you ask something specific like "I want to be an astronaut, what can I learn", only a few tight clusters glow. But ask "I'm bored, what could be something fun to study" and clusters across the space light up. #applevisionpro #artificialintelligence #xr #rag

Imagine stepping into your embeddings that power RAG. 📚 In this proof of concept, you’re surrounded by courses. Semantically similar ones float right next to each other. I've built an XR prototype with Rémi at In The Pocket where you can stand inside your knowledge base on Apple Vision Pro. Ask a question and the AI lights up the relevant documents in space, so you literally see where your answer comes from. When you ask something specific like "I want to be an astronaut, what can I learn", only a few tight clusters glow. But ask "I'm bored, what could be something fun to study" and clusters across the space light up. #applevisionpro #artificialintelligence #xr #rag

Stijn Spanhove

24,747 views • 4 months ago

If you’re not at Daytona, you can’t get closer to the action than this! Experience the Daytona 500 on the FOX Sports XR app for the Galaxy XR headset powered by Android XR or with a live immersive experience at the Cosm Dallas venue.

If you’re not at Daytona, you can’t get closer to the action than this! Experience the Daytona 500 on the FOX Sports XR app for the Galaxy XR headset powered by Android XR or with a live immersive experience at the Cosm Dallas venue.

FOX: NASCAR

16,346 views • 4 months ago

We asked Gemini 3.5 Flash to bring back the classic early-2000s PC drawing experience, and it delivered in one shot. What are you building first with Canvas?

We asked Gemini 3.5 Flash to bring back the classic early-2000s PC drawing experience, and it delivered in one shot. What are you building first with Canvas?

Google Gemini

22,289,817 views • 27 days ago

FreeTimeGS – Free Gaussian Primitives Anywhere, Anytime – has been making waves online over the past two days. You may have already seen it. If not, here’s a great example: a live demo with minimal cleanup and no heavy post-production. What’s remarkable is the simplicity of their setup. This example shows a basic semicircle rig made from smartphones and budget light stands. No complex volumetric or photogrammetry system required—and yet, the result plays smoothly right in a mobile browser. Yes, even on my iPhone. You can also view this on Vision Pro! The next logical step? Let’s bring this into the new Spatial Web standard IEEE IEEE 2874 and make it viewable in Apple Vision Pro, Android XR, and other upcoming headsets via open source XR web browser. Imagine freely flying through AI-powered Gaussian reconstructions directly in spatial environments cross platforms. We’re witnessing a major leap in spatial computing. AI-powered tools like this are reshaping every part of the creative pipeline. Can't wait to see what Apple has to say tomorrow at WWDC 25! Are you ready for it? What use cases excite you most? Drop your thoughts below—let’s discuss.

FreeTimeGS – Free Gaussian Primitives Anywhere, Anytime – has been making waves online over the past two days. You may have already seen it. If not, here’s a great example: a live demo with minimal cleanup and no heavy post-production. What’s remarkable is the simplicity of their setup. This example shows a basic semicircle rig made from smartphones and budget light stands. No complex volumetric or photogrammetry system required—and yet, the result plays smoothly right in a mobile browser. Yes, even on my iPhone. You can also view this on Vision Pro! The next logical step? Let’s bring this into the new Spatial Web standard IEEE IEEE 2874 and make it viewable in Apple Vision Pro, Android XR, and other upcoming headsets via open source XR web browser. Imagine freely flying through AI-powered Gaussian reconstructions directly in spatial environments cross platforms. We’re witnessing a major leap in spatial computing. AI-powered tools like this are reshaping every part of the creative pipeline. Can't wait to see what Apple has to say tomorrow at WWDC 25! Are you ready for it? What use cases excite you most? Drop your thoughts below—let’s discuss.

Hugh Hou

12,665 views • 1 year ago

Go to NASA's YouTube channel right now on your Apple Vision Pro or Samsung Galaxy XR. You can watch the Artemis II launch in 8K 360. It feels like you're standing at Kennedy Space Center as the rocket takes off.

Go to NASA's YouTube channel right now on your Apple Vision Pro or Samsung Galaxy XR. You can watch the Artemis II launch in 8K 360. It feels like you're standing at Kennedy Space Center as the rocket takes off.

Spatial Insider

233,752 views • 2 months ago

With Gemini 3 in Search, you can now create interactive tools and simulations to visualize complex topics. It’s been really great to see the creative ways people are using it for things like seeing how photosynthesis works or how a vinyl record produces sound. Here are some of my favorite examples of what Gemini 3 in AI Mode can do:

With Gemini 3 in Search, you can now create interactive tools and simulations to visualize complex topics. It’s been really great to see the creative ways people are using it for things like seeing how photosynthesis works or how a vinyl record produces sound. Here are some of my favorite examples of what Gemini 3 in AI Mode can do:

Rajan Patel

26,298 views • 7 months ago

Really happy to see the interest around our “Hands-on with Gemini” video. In our developer blog yesterday, we broke down how Gemini was used to create it. We gave Gemini sequences of different modalities — image and text in this case — and had it respond by predicting what might come next. Devs can try similar things when access to Pro opens on 12/13 🚀. The knitting demo used Ultra⚡ All the user prompts and outputs in the video are real, shortened for brevity. The video illustrates what the multimodal user experiences built with Gemini could look like. We made it to inspire developers. When you’re building an app, you can get similar results (there’s always some variability with LLMs) by prompting Gemini with an instruction that allows the user to "configure" the behavior of the model, like inputting “you are an expert in science …” before a user can engage in the same kind of back and forth dialogue. Here’s a clip of what this looks like in AI Studio with Gemini Pro. We’ve come a long way since Flamingo 🦩 & PALI, looking forward to seeing what people build with it!

Really happy to see the interest around our “Hands-on with Gemini” video. In our developer blog yesterday, we broke down how Gemini was used to create it. We gave Gemini sequences of different modalities — image and text in this case — and had it respond by predicting what might come next. Devs can try similar things when access to Pro opens on 12/13 🚀. The knitting demo used Ultra⚡ All the user prompts and outputs in the video are real, shortened for brevity. The video illustrates what the multimodal user experiences built with Gemini could look like. We made it to inspire developers. When you’re building an app, you can get similar results (there’s always some variability with LLMs) by prompting Gemini with an instruction that allows the user to "configure" the behavior of the model, like inputting “you are an expert in science …” before a user can engage in the same kind of back and forth dialogue. Here’s a clip of what this looks like in AI Studio with Gemini Pro. We’ve come a long way since Flamingo 🦩 & PALI, looking forward to seeing what people build with it!

Oriol Vinyals

180,952 views • 2 years ago

Google is on 🔥🔥 They have announced a new set of updates: - New feature called personalization, powered by Gemini 2.0 Flash Thinking Experimental . - Gemini Advanced users will have access to a 1M token context window with 2.0 Flash Thinking Experimental . - Deep research with Gemini 2.0 Flash Thinking Experimental . - Gems are now available for everyone, including the free tier.

Google is on 🔥🔥 They have announced a new set of updates: - New feature called personalization, powered by Gemini 2.0 Flash Thinking Experimental . - Gemini Advanced users will have access to a 1M token context window with 2.0 Flash Thinking Experimental . - Deep research with Gemini 2.0 Flash Thinking Experimental . - Gems are now available for everyone, including the free tier.

AshutoshShrivastava

31,939 views • 1 year ago

It's OVER... The new Gemini 2.5 Pro is insane! Here are some WILD examples that I've been able to create in minutes...

It's OVER... The new Gemini 2.5 Pro is insane! Here are some WILD examples that I've been able to create in minutes...

Wes Roth

15,953 views • 1 year ago