Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Time to update my Quest MR + Gen AI prototype now that Meta Quest Camera API is out — This proto is using a workaround to capture passthrough snapshots, processes them with Stable Diffusion, and displays the results back in their exact physical location. #MR #AI

smallfly's profile picture

Hugues Bruyère

7,296 subscribers

11,744 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & Technologie#MR#AI
LIVE

Anya Rossi• Live Now

Private livecam show

9 Kommentare

Profilbild von VRCatalans
VRCatalansvor 1 Jahr

@MetaQuestVR Do you think it would be possible to project textures on the scanned geometry of the environment? I'm thinking of these games like the rollercoaster game where you switched to VR and visited your room scan without textures

Profilbild von Hugues Bruyère
Hugues Bruyèrevor 1 Jahr

@MetaQuestVR Yes, I do not see why it would not be possible.

Profilbild von PowerBeatsVR
PowerBeatsVRvor 3 Jahren

Get ready for a full-body VR workout that’s fun, fast, and intuitive — Play PowerBeatsVR (Now on Meta Quest) 🔥

Profilbild von VentureMind AI
VentureMind AIvor 1 Jahr

@MetaQuestVR Cool tek!

Profilbild von أمين رعيان
أمين رعيانvor 1 Jahr

@MetaQuestVR + this ?

Profilbild von Hugues Bruyère
Hugues Bruyèrevor 1 Jahr

@MetaQuestVR This (or similar) is the next step. I used StreamDiffusion in the past. 

Profilbild von Saurs
Saursvor 1 Jahr

@MetaQuestVR Seen alot of stuff like this lately, and its been boring from the get go, it needs to be more special. It's so ass as it is, you're just img2img through SD api and nothing else. Please find something special with it, not the basics

Profilbild von Hugues Bruyère
Hugues Bruyèrevor 1 Jahr

@MetaQuestVR Just the first step of an iterative process.

Profilbild von treek
treekvor 1 Jahr

@MetaQuestVR Camera API? Huhhhh?

Ähnliche Videos

In this iteration, I’ve added AI at Meta’s SAM 2 to our workflow to segment people. Lisa and I had fun testing it in our lab—striking poses in sequence, with our bodies precisely isolated for a refined Stable Diffusion effect. #MR #AI #Passthrough Meta for Developers
0:39

In this iteration, I’ve added AI at Meta’s SAM 2 to our workflow to segment people. Lisa and I had fun testing it in our lab—striking poses in sequence, with our bodies precisely isolated for a refined Stable Diffusion effect. #MR #AI #Passthrough Meta for Developers

Hugues Bruyère

32,762 Aufrufe • vor 1 Jahr

It won't take long before we'll be able to "reskin reality" in real-time. I had the chance to try this prototype that combines the #MixedReality view on a Quest device with Stable Diffusion AI and it feels like all the pieces are about to fit together... Special thanks to Hugues Bruyère , Chief of Innovation at Dpt. for sharing this early version.
0:36

It won't take long before we'll be able to "reskin reality" in real-time. I had the chance to try this prototype that combines the #MixedReality view on a Quest device with Stable Diffusion AI and it feels like all the pieces are about to fit together... Special thanks to Hugues Bruyère , Chief of Innovation at Dpt. for sharing this early version.

Gabriele Romagnoli

33,899 Aufrufe • vor 1 Jahr

It‘s real if you can dream it. A quest to explore the limits of AI-powered storytelling coming soon. This trailer is a result of a true co:creation between human creative minds and powerful AI tools like Midjourney, Stable Diffusion, AIVA & Synthesys AI. #whitemirror
0:52

It‘s real if you can dream it. A quest to explore the limits of AI-powered storytelling coming soon. This trailer is a result of a true co:creation between human creative minds and powerful AI tools like Midjourney, Stable Diffusion, AIVA & Synthesys AI. #whitemirror

White Mirror

71,891 Aufrufe • vor 3 Jahren

Now it’s clear why Meta rebranded to MR. My kid played an alpha version of DreamPark for 2 hours straight—AR gaming on headsets is a 10x experience. Unlocking camera access for devs is the biggest ‘why now’ moment for Quest.
0:22

Now it’s clear why Meta rebranded to MR. My kid played an alpha version of DreamPark for 2 hours straight—AR gaming on headsets is a 10x experience. Unlocking camera access for devs is the biggest ‘why now’ moment for Quest.

Hubert Thieblot

41,641 Aufrufe • vor 1 Jahr

Here is the new fullscreen video player UX for Meta Quest Browser that is currently rolling out. Honestly a massive improvement. One quirk is that the environment dimming slider doesn't seem to work in passthrough, might be a bug though.
0:20

Here is the new fullscreen video player UX for Meta Quest Browser that is currently rolling out. Honestly a massive improvement. One quirk is that the environment dimming slider doesn't seem to work in passthrough, might be a bug though.

Luna

51,931 Aufrufe • vor 2 Jahren

Exploding Kittens is coming to VR! 🕹 Three unique game modes 🗯 Engage with others in the immersive social hub 💥 For up to 5 k̵i̵t̵t̵e̵n̵s̵ human players Pre-order now: Coming to the Meta Quest 2 and Meta Quest 3 later this year.
0:55

Exploding Kittens is coming to VR! 🕹 Three unique game modes 🗯 Engage with others in the immersive social hub 💥 For up to 5 k̵i̵t̵t̵e̵n̵s̵ human players Pre-order now: Coming to the Meta Quest 2 and Meta Quest 3 later this year.

Saber Interactive

475,089 Aufrufe • vor 2 Jahren

Building virtual sets on top of reality, moving around in them and seeing them transformed entirely with Gen AI. Is this the future? :D This was quite simple to make actually: Figmin XR running on my Quest 3, using some models from sketchfab and some custom creations to set-dress my hallway. Filming a small performance of walking through the set and then using Runway Gen 3 to transform that. I'm still missing a lot of control with the AI process, like uploading style frames along the duration of the video.. but overall this is pretty incredible! Can't help but think about how insane it would have been to be able to use Dreams for stuff like this! What do you think? #art #ai
0:34

Building virtual sets on top of reality, moving around in them and seeing them transformed entirely with Gen AI. Is this the future? :D This was quite simple to make actually: Figmin XR running on my Quest 3, using some models from sketchfab and some custom creations to set-dress my hallway. Filming a small performance of walking through the set and then using Runway Gen 3 to transform that. I'm still missing a lot of control with the AI process, like uploading style frames along the duration of the video.. but overall this is pretty incredible! Can't help but think about how insane it would have been to be able to use Dreams for stuff like this! What do you think? #art #ai

Martin Nebelong

81,989 Aufrufe • vor 1 Jahr

3d gaussian splatting is fucking cool. And now you can capture real world spaces just by walking around in your meta quest headset. The real-time feedback ensures you don’t miss a spot. Apple needs to get on this asap:
1:03

3d gaussian splatting is fucking cool. And now you can capture real world spaces just by walking around in your meta quest headset. The real-time feedback ensures you don’t miss a spot. Apple needs to get on this asap:

Bilawal Sidhu

162,919 Aufrufe • vor 10 Monaten

"The music video tells a story of what life would be like if we were to change things or make an insignificant choice. The power of AI and stable diffusion lies in its ability to transport Peter to any location, dress in any fashion and experience dimensions beyond the real world." - Junie Lau Junie's Official Video for Peter's 'The Court' (Dark-Side Mix) is out now! Stability AI Stable Diffusion #diffusetogether
0:30

"The music video tells a story of what life would be like if we were to change things or make an insignificant choice. The power of AI and stable diffusion lies in its ability to transport Peter to any location, dress in any fashion and experience dimensions beyond the real world." - Junie Lau Junie's Official Video for Peter's 'The Court' (Dark-Side Mix) is out now! Stability AI Stable Diffusion #diffusetogether

Peter Gabriel

74,251 Aufrufe • vor 2 Jahren

In the past, AI motion capture was too unstable to form a reliable workflow. Now, with a 1-billion-parameter AI real-time motion model, you can connect a 1080P camera or upload videos to achieve highly stable, low-latency real-time capture. It supports UE, Unity, and Blender, and requires 8GB of VRAM for real-time processing.
3:14

In the past, AI motion capture was too unstable to form a reliable workflow. Now, with a 1-billion-parameter AI real-time motion model, you can connect a 1080P camera or upload videos to achieve highly stable, low-latency real-time capture. It supports UE, Unity, and Blender, and requires 8GB of VRAM for real-time processing.

CYANPUPPETS

18,703 Aufrufe • vor 3 Monaten

this spreads unnecessary fears and misconceptions about ai. studios develop their own ai tools to help speedup processes that are tedious and time consuming. klaus is another example that used ai in order to achieve their unique style
1:30

this spreads unnecessary fears and misconceptions about ai. studios develop their own ai tools to help speedup processes that are tedious and time consuming. klaus is another example that used ai in order to achieve their unique style

Z

2,407,429 Aufrufe • vor 2 Jahren

Puzzle Quest: Immortal Edition is out now! Featuring all-new content, including a new class, and upgraded visuals to 4K, this is the definitive Puzzle Quest experience. Are you ready to save the kingdom of Etheria? It's time to find out!
0:32

Puzzle Quest: Immortal Edition is out now! Featuring all-new content, including a new class, and upgraded visuals to 4K, this is the definitive Puzzle Quest experience. Are you ready to save the kingdom of Etheria? It's time to find out!

Epic Games Store

27,984 Aufrufe • vor 10 Monaten

With our latest advances in Meta Llama 3, Meta AI is now smarter and better than ever. You can use Meta AI in Facebook, Instagram, Messenger and WhatsApp to get things done and access real-time info without having to leave the app you’re using.
0:33

With our latest advances in Meta Llama 3, Meta AI is now smarter and better than ever. You can use Meta AI in Facebook, Instagram, Messenger and WhatsApp to get things done and access real-time info without having to leave the app you’re using.

Meta Newsroom

10,311 Aufrufe • vor 2 Jahren

Vertigo Rush is OUT NOW on Meta Quest! 🏁 - Race against AI - Climb the global Time Trial leaderboards - Play online with up to 10 friends Speed through visually stunning, handcrafted tracks Grab it now at our exclusive Early Access launch price!
0:24

Vertigo Rush is OUT NOW on Meta Quest! 🏁 - Race against AI - Climb the global Time Trial leaderboards - Play online with up to 10 friends Speed through visually stunning, handcrafted tracks Grab it now at our exclusive Early Access launch price!

Sumalab Studio

121,502 Aufrufe • vor 19 Tagen

My Pub world is Quest Compatible with all its mini games!! I have also started to work on a quest Laserdome for #VRChat and after a few quest projects I want to request a feature to make things easier for PC/Quest games that are built at the same time.
1:27

My Pub world is Quest Compatible with all its mini games!! I have also started to work on a quest Laserdome for #VRChat and after a few quest projects I want to request a feature to make things easier for PC/Quest games that are built at the same time.

xCirrex

13,243 Aufrufe • vor 3 Monaten

🚨Ray-Ban & Oakley Meta Glasses update 🚨 -Faster camera Our engineers are cracked and found a way to make media capture latency even faster with no compromises. You're welcome. - Calendar You can now connect your Google and Outlook calendars to Meta AI to receive event notifications, create personal events, and find important calendar information just by asking Meta AI. For example, say “Hey Meta, what time is my first meeting tomorrow morning?” or “Hey Meta, add yoga to my calendar tomorrow at 9 AM.” - Audible voice search expanded availability You can now use voice commands (e.g., "Hey Meta, play my audiobook") to control Audible in all English-speaking regions where both the Audible and Meta AI apps are available. - Creation location for reminders If you've turned on location services for the Meta AI app, the creation location will be saved when you create a reminder. This allows you to see the location where a reminder was created in the Meta AI app. You can also ask Meta AI about it. - Finally, in recent months, we've began to run a lot more A/B tests and experiments, so you may be seeing some other nuggets. Will shout them out when we learn and validate these on a wider scale. If you haven't already, consider opting into our Early Access Program (EAP) in Settings. As you can guess, many more cool announcements coming next month at Connect, so enjoy this POV footage of me practicing with my little QB and stay tuned!
0:38

🚨Ray-Ban & Oakley Meta Glasses update 🚨 -Faster camera Our engineers are cracked and found a way to make media capture latency even faster with no compromises. You're welcome. - Calendar You can now connect your Google and Outlook calendars to Meta AI to receive event notifications, create personal events, and find important calendar information just by asking Meta AI. For example, say “Hey Meta, what time is my first meeting tomorrow morning?” or “Hey Meta, add yoga to my calendar tomorrow at 9 AM.” - Audible voice search expanded availability You can now use voice commands (e.g., "Hey Meta, play my audiobook") to control Audible in all English-speaking regions where both the Audible and Meta AI apps are available. - Creation location for reminders If you've turned on location services for the Meta AI app, the creation location will be saved when you create a reminder. This allows you to see the location where a reminder was created in the Meta AI app. You can also ask Meta AI about it. - Finally, in recent months, we've began to run a lot more A/B tests and experiments, so you may be seeing some other nuggets. Will shout them out when we learn and validate these on a wider scale. If you haven't already, consider opting into our Early Access Program (EAP) in Settings. As you can guess, many more cool announcements coming next month at Connect, so enjoy this POV footage of me practicing with my little QB and stay tuned!

David Woodland

53,750 Aufrufe • vor 11 Monaten

META launches a new feature that will allow Facebook and Instagram to continuously scan all your photos and video and upload them to their cloud “Facebook is now using AI to scan every single photo and video within your phone — This is terrifying. Not only are they scanning those photos and videos, but then they're uploading it to the cloud continuously. It's operating continuously in the background scanning those photos to figure out the time of day, the location, the other people in your photos, that it can determine what it thinks you should post” The scanning happens on-device using AI models for suggestions, but Meta can upload and process data to the cloud when you interact It’s in opt in feature, so make sure to never opt in and if you have make sure to opt out The service is called ‘Camera Roll Sharing Suggestions’
2:10

META launches a new feature that will allow Facebook and Instagram to continuously scan all your photos and video and upload them to their cloud “Facebook is now using AI to scan every single photo and video within your phone — This is terrifying. Not only are they scanning those photos and videos, but then they're uploading it to the cloud continuously. It's operating continuously in the background scanning those photos to figure out the time of day, the location, the other people in your photos, that it can determine what it thinks you should post” The scanning happens on-device using AI models for suggestions, but Meta can upload and process data to the cloud when you interact It’s in opt in feature, so make sure to never opt in and if you have make sure to opt out The service is called ‘Camera Roll Sharing Suggestions’

Wall Street Apes

218,358 Aufrufe • vor 19 Tagen

Stable Diffusion 3.5 from Stability AI is now LIVE on Civitai! Stable Diffusion 3.5 Large is a Multimodal Diffusion Transformer text-to-image model that features improved performance in image quality, typography, complex prompt understanding, and resource-efficiency. Generate with SD 3.5 Live on Civitai here👇🏽 All images used to create this video were generated with SD 3.5 by our community 🫶🏽
0:25

Stable Diffusion 3.5 from Stability AI is now LIVE on Civitai! Stable Diffusion 3.5 Large is a Multimodal Diffusion Transformer text-to-image model that features improved performance in image quality, typography, complex prompt understanding, and resource-efficiency. Generate with SD 3.5 Live on Civitai here👇🏽 All images used to create this video were generated with SD 3.5 by our community 🫶🏽

Civitai

31,428 Aufrufe • vor 1 Jahr

Passthrough Camera access (PCA) on Quest may have taken a while and I already see some devs doubting the impact this will have for XR. I think the most fun thing about PCA is, that there are such smart devs out there, that we cannot even begin to imagine what some people will come up with, and I am very much looking forward to that. Below is another sample I made and included in my QuestCameraKit GitHub repo. It is using API calls to OpenAI vision and whisper and is capable of understanding and responding in virtually any language (English and German below):
0:45

Passthrough Camera access (PCA) on Quest may have taken a while and I already see some devs doubting the impact this will have for XR. I think the most fun thing about PCA is, that there are such smart devs out there, that we cannot even begin to imagine what some people will come up with, and I am very much looking forward to that. Below is another sample I made and included in my QuestCameraKit GitHub repo. It is using API calls to OpenAI vision and whisper and is capable of understanding and responding in virtually any language (English and German below):

Robi 🔜 Meta Connect ᯅ

11,831 Aufrufe • vor 1 Jahr

Postman is going all-in on AI! Their AI Agent Builder is live (and it's pretty powerful!) Postman's API Network has 100,000+ APIs and they just released a suite of tools for building AI Agents that can automatically discover and connect to any of these APIs. Think about it for a second: You can now build an AI agent that searches the network, discovers an API, processes its documentation, connects to it, and evaluates whether everything worked at the end of the process. This is a huge deal. Every person I know uses the Postman client to work with APIs. Now, they will have access to the AI Agent Builder, a complete suite for designing, building, testing, and deploying agents. I've built a few simple agents myself, and it's tough. The ability to discover and use an API automatically is really exciting (and a huge time saver if it works as promised!) In collaboration with the Postman team, who is sponsoring this post, I recorded a video with my thoughts about the Postman AI Agent Builder.
3:38

Postman is going all-in on AI! Their AI Agent Builder is live (and it's pretty powerful!) Postman's API Network has 100,000+ APIs and they just released a suite of tools for building AI Agents that can automatically discover and connect to any of these APIs. Think about it for a second: You can now build an AI agent that searches the network, discovers an API, processes its documentation, connects to it, and evaluates whether everything worked at the end of the process. This is a huge deal. Every person I know uses the Postman client to work with APIs. Now, they will have access to the AI Agent Builder, a complete suite for designing, building, testing, and deploying agents. I've built a few simple agents myself, and it's tough. The ability to discover and use an API automatically is really exciting (and a huge time saver if it works as promised!) In collaboration with the Postman team, who is sponsoring this post, I recorded a video with my thoughts about the Postman AI Agent Builder.

Santiago

230,594 Aufrufe • vor 1 Jahr