Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

New paper from Apple - Sharp Monocular View Synthesis in Less than a Second Mescheder et al. @ Apple just released a very impressive paper (congrats! 🎉🥳). You give it an image and it generates a really great looking 3d Gaussian representation. Uses depth pro. It's really good. The... show more

Tim Davison ᯅ

4,960 subscribers

132,924 views • 8 months ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

People are underestimating Apple in AI. I just ran Apple’s new SHARP model locally and watched my photos turn into 3D Gaussian splats in seconds, then stepped inside them on Vision Pro. This feels like the beginning of something special. You really have to try it.

People are underestimating Apple in AI. I just ran Apple’s new SHARP model locally and watched my photos turn into 3D Gaussian splats in seconds, then stepped inside them on Vision Pro. This feels like the beginning of something special. You really have to try it.

Justin Ryan

255,483 views • 8 months ago

Submerged—the first scripted short film captured in Apple Immersive Video—is available on Apple Vision Pro. And, wow, it’s awesome! Edward Berger takes viewers on an action-packed story from inside a submarine. Go behind-the-scenes this weekend:

Submerged—the first scripted short film captured in Apple Immersive Video—is available on Apple Vision Pro. And, wow, it’s awesome! Edward Berger takes viewers on an action-packed story from inside a submarine. Go behind-the-scenes this weekend:

Greg Joswiak

53,234 views • 1 year ago

From 2D video to a very rough 4D Gaussian Splatting POC on the Apple Vision Pro 👀 Took Big Buck Bunny frames, turned them into Gaussian splats with Sharp (Apple), compressed them with PlayCanvas splat transform to SOG, and played all frames via WebXR + spark Still rough and not fully optimized, but wow, dynamic GS in AVP feels wild 🤯 Second part of the video is on my laptop #threejs

From 2D video to a very rough 4D Gaussian Splatting POC on the Apple Vision Pro 👀 Took Big Buck Bunny frames, turned them into Gaussian splats with Sharp (Apple), compressed them with PlayCanvas splat transform to SOG, and played all frames via WebXR + spark Still rough and not fully optimized, but wow, dynamic GS in AVP feels wild 🤯 Second part of the video is on my laptop #threejs

Stijn Spanhove

20,127 views • 8 months ago

WOW. 😳 Apple just quietly won the 3D maps war at WWDC. Gaussian Splatting is coming to Apple Maps Flyover this fall. Apple Maps Flyover covers 300+ cities. Until yesterday, every single one was built on standard drone photogrammetry. The technology captures photos from the air and reconstructs 3D geometry from them. Gaussian Splatting does not reconstruct geometry. It represents the scene as millions of tiny 3D ellipsoids, each one carrying its own color and opacity information based on how light actually behaves in that location. The output is not a mesh model. It is a field of light. When you move through it, it does not crumble at the edges. The detail holds because it was never geometry to begin with. Apple has been hiring for this for years. Their SHARP model, published in research last year, generates photorealistic 3D scenes from a single image in under a second. Google has more sensor data than anyone. More Street View cars, more satellites, more capture history. On navigation accuracy and geodata depth, Google Maps is still ahead by most measures. But fidelity in 3D city rendering is a different competition, and Apple just set a bar in that. Most people will experience this in the fall without knowing the name of the technology. They will open Flyover, look at a city they know, and notice it looks different. Real, not rendered. That is the moment Gaussian Splatting stops being a research term and becomes something a billion people use. Bookmark this. It will look prescient by October.

WOW. 😳 Apple just quietly won the 3D maps war at WWDC. Gaussian Splatting is coming to Apple Maps Flyover this fall. Apple Maps Flyover covers 300+ cities. Until yesterday, every single one was built on standard drone photogrammetry. The technology captures photos from the air and reconstructs 3D geometry from them. Gaussian Splatting does not reconstruct geometry. It represents the scene as millions of tiny 3D ellipsoids, each one carrying its own color and opacity information based on how light actually behaves in that location. The output is not a mesh model. It is a field of light. When you move through it, it does not crumble at the edges. The detail holds because it was never geometry to begin with. Apple has been hiring for this for years. Their SHARP model, published in research last year, generates photorealistic 3D scenes from a single image in under a second. Google has more sensor data than anyone. More Street View cars, more satellites, more capture history. On navigation accuracy and geodata depth, Google Maps is still ahead by most measures. But fidelity in 3D city rendering is a different competition, and Apple just set a bar in that. Most people will experience this in the fall without knowing the name of the technology. They will open Flyover, look at a city they know, and notice it looks different. Real, not rendered. That is the moment Gaussian Splatting stops being a research term and becomes something a billion people use. Bookmark this. It will look prescient by October.

Shruti

19,832 views • 2 months ago

Just experienced streamable 4D Gaussian Splatting content on Apple Vision Pro… and it's insane 🤯 Seeing life sized volumetric people performing right in my room is kind of mind blowing. It really feels like real humans are standing in front of you. And it streams online just like regular video. No massive downloads. No waiting. Awesome work by Gracia 👏 Apple Vision Pro + Gaussian Splatting is one of the most futuristic things I’ve seen in a long time.

Just experienced streamable 4D Gaussian Splatting content on Apple Vision Pro… and it's insane 🤯 Seeing life sized volumetric people performing right in my room is kind of mind blowing. It really feels like real humans are standing in front of you. And it streams online just like regular video. No massive downloads. No waiting. Awesome work by Gracia 👏 Apple Vision Pro + Gaussian Splatting is one of the most futuristic things I’ve seen in a long time.

Ben Geskin

1,011,993 views • 3 months ago

From 2 second (faster on higher end GPUs) Gaussian splat generations, using Apple's "Sharp Monocular View Synthesis in Less Than a Second" code, running locally on my system. Imported into Octane Render 2026 which has fully path traced rendering of Gaussian splats. Stuff like this is why I've been so excited to see where Gaussian splatting research would go. Combined with a real-time Generative AI "renderer" this stuff could start to get quite insane. Combined with Dreams it would be even better ofc :D Code to the Apple code in the comments below.

From 2 second (faster on higher end GPUs) Gaussian splat generations, using Apple's "Sharp Monocular View Synthesis in Less Than a Second" code, running locally on my system. Imported into Octane Render 2026 which has fully path traced rendering of Gaussian splats. Stuff like this is why I've been so excited to see where Gaussian splatting research would go. Combined with a real-time Generative AI "renderer" this stuff could start to get quite insane. Combined with Dreams it would be even better ofc :D Code to the Apple code in the comments below.

Martin Nebelong

36,774 views • 8 months ago

🚀 Just created something completely new! I Rewrote Apple's Sharp to generate 3D Gaussian Splats from a single 360° equirectangular image made from FLUX 2 (Flex) One equirectangular image → Full spherical 3DGS world! ✨ New web interface - drag & drop your pano, select quality, download PLY ⚡ 2-3 sec on a older laptop Quality tiers: ▫️ Low: 500K splats ▫️ Medium: 2M splats ▫️ High: 4.7M splats ▫️ Ultra: 8.4M splats The pipeline: 360° depth estimation from equirectangular input Spherical projection maps pixels → 3D points Each point becomes a Gaussian with depth-scaled radius - Export to standard PLY for any 3DGS viewer (Unity) No multi-view capture. No COLMAP. No NeRF training. Just upload a pano, pick quality, get your splats. #3DGS #GaussianSplatting #ComputerVision #AI #madewithunity #Splats Sharp Monocular View Synthesis in Less Than a Second

🚀 Just created something completely new! I Rewrote Apple's Sharp to generate 3D Gaussian Splats from a single 360° equirectangular image made from FLUX 2 (Flex) One equirectangular image → Full spherical 3DGS world! ✨ New web interface - drag & drop your pano, select quality, download PLY ⚡ 2-3 sec on a older laptop Quality tiers: ▫️ Low: 500K splats ▫️ Medium: 2M splats ▫️ High: 4.7M splats ▫️ Ultra: 8.4M splats The pipeline: 360° depth estimation from equirectangular input Spherical projection maps pixels → 3D points Each point becomes a Gaussian with depth-scaled radius - Export to standard PLY for any 3DGS viewer (Unity) No multi-view capture. No COLMAP. No NeRF training. Just upload a pano, pick quality, get your splats. #3DGS #GaussianSplatting #ComputerVision #AI #madewithunity #Splats Sharp Monocular View Synthesis in Less Than a Second

Daniel Skaale

73,120 views • 7 months ago

Apple’s new Gaussian Splat model seems like the perfect tech for a native Maps app on Vision Pro. Imagine an immersive version of Look Around where you can actually walk through cities and explore places from every angle, in 3D. I tried it with some of my favorite places, and it genuinely feels like being there again. It’s wild that this works from just a single image.

Apple’s new Gaussian Splat model seems like the perfect tech for a native Maps app on Vision Pro. Imagine an immersive version of Look Around where you can actually walk through cities and explore places from every angle, in 3D. I tried it with some of my favorite places, and it genuinely feels like being there again. It’s wild that this works from just a single image.

Phil Traut ᯅ

31,958 views • 8 months ago

Metallica announces an Apple Immersive concert experience, available for free only on Apple Vision Pro starting March 14. An extended preview will be available during Apple Vision Pro demos in select Apple Stores. "Apple built a custom stage plot featuring 14 Apple Immersive Video cameras using a mix of stabilized cameras, cable-suspended cameras, and remote-controlled camera dolly systems that moved around the stage." "Filmed during our final M72 stop of 2024 in Mexico City, it features full performances of “Whiplash,” “One,” and “Enter Sandman,” all captured exclusively in Apple Immersive Video." "The songs featured in Metallica will also be available as a new EP, Metallica Live from Mexico City, in Spatial Audio on Apple Music this Friday." "An extended preview of Metallica will be available as part of the Apple Vision Pro demo in Apple Store locations worldwide beginning Friday, March 14. Apple Vision Pro is available in Australia, Canada, France, Germany, Hong Kong, Japan, Singapore, South Korea, Taiwan, the United Arab Emirates, the United Kingdom, and the United States. Customers in those countries can reserve a free Vision Pro demo at their local Apple Store. You must have an Apple account to schedule."

Metallica announces an Apple Immersive concert experience, available for free only on Apple Vision Pro starting March 14. An extended preview will be available during Apple Vision Pro demos in select Apple Stores. "Apple built a custom stage plot featuring 14 Apple Immersive Video cameras using a mix of stabilized cameras, cable-suspended cameras, and remote-controlled camera dolly systems that moved around the stage." "Filmed during our final M72 stop of 2024 in Mexico City, it features full performances of “Whiplash,” “One,” and “Enter Sandman,” all captured exclusively in Apple Immersive Video." "The songs featured in Metallica will also be available as a new EP, Metallica Live from Mexico City, in Spatial Audio on Apple Music this Friday." "An extended preview of Metallica will be available as part of the Apple Vision Pro demo in Apple Store locations worldwide beginning Friday, March 14. Apple Vision Pro is available in Australia, Canada, France, Germany, Hong Kong, Japan, Singapore, South Korea, Taiwan, the United Arab Emirates, the United Kingdom, and the United States. Customers in those countries can reserve a free Vision Pro demo at their local Apple Store. You must have an Apple account to schedule."

M1

15,476 views • 1 year ago

Okay, this is absolutely WILD – Apple’s new Sharp model is even more impressive on Vision Pro when you place it exactly where the original photo was taken. Since we’ll probably never have snow on Christmas again, seeing my parents’ garden covered in snow once more made me really emotional. I still can’t believe this was created from a single picture. We’re getting very close to real-life time machines.

Okay, this is absolutely WILD – Apple’s new Sharp model is even more impressive on Vision Pro when you place it exactly where the original photo was taken. Since we’ll probably never have snow on Christmas again, seeing my parents’ garden covered in snow once more made me really emotional. I still can’t believe this was created from a single picture. We’re getting very close to real-life time machines.

Phil Traut ᯅ

57,376 views • 8 months ago

.GeoGuessr is a geography game in which players have to guess a location from a Google Maps street view image. It can be anywhere on the planet and some players can figure it out in less than a second. Nancy Chen met up with some of the top players.

.GeoGuessr is a geography game in which players have to guess a location from a Google Maps street view image. It can be anywhere on the planet and some players can figure it out in less than a second. Nancy Chen met up with some of the top players.

CBS Saturday Morning

45,165 views • 1 year ago

I tried Meta’s new SAM 3D tool. It’s impressive. In about ten seconds it segmented the bed from my photo and turned it into a full 3D object. The quality surprised me. Only catch right now is it exports as PLY. I’d love USDZ so I can drop it straight into my room on Vision Pro.

I tried Meta’s new SAM 3D tool. It’s impressive. In about ten seconds it segmented the bed from my photo and turned it into a full 3D object. The quality surprised me. Only catch right now is it exports as PLY. I’d love USDZ so I can drop it straight into my room on Vision Pro.

Justin Ryan

52,050 views • 9 months ago

Now playing: My new podcast, "The Movie of Your Life." The "pilot" episode is available now on Apple, Spotify, et al., and we have some great guests lined up in the weeks to come! Hope you'll give it a spin.

Now playing: My new podcast, "The Movie of Your Life." The "pilot" episode is available now on Apple, Spotify, et al., and we have some great guests lined up in the weeks to come! Hope you'll give it a spin.

Richard Roeper

21,079 views • 10 months ago

This is insane!🔥 This is a 720p (2D) video I shot of my kids 15 years ago! I used AI to upscale it, depth-map it, & used the depth map to convert it to 3D SBS, & then converted that to Spatial Video for Apple Vision Pro (MV-HEVC). My son is currently a senior in high school, and my daughter is a sophomore in college. This video has complete spatial depth when viewed in the Apple Vision Pro. Obviously, you can't appreciate that via this post. But it's like I'm there & I can see into this room. Talk about giving you the feels.🥹 This can be done with ANY video. 🤯 I'm working on refining the workflow, as it's a bit cumbersome. I plan to share it, but holy cow! I had to post this.

This is insane!🔥 This is a 720p (2D) video I shot of my kids 15 years ago! I used AI to upscale it, depth-map it, & used the depth map to convert it to 3D SBS, & then converted that to Spatial Video for Apple Vision Pro (MV-HEVC). My son is currently a senior in high school, and my daughter is a sophomore in college. This video has complete spatial depth when viewed in the Apple Vision Pro. Obviously, you can't appreciate that via this post. But it's like I'm there & I can see into this room. Talk about giving you the feels.🥹 This can be done with ANY video. 🤯 I'm working on refining the workflow, as it's a bit cumbersome. I plan to share it, but holy cow! I had to post this.

Blaine Brown

1,468,678 views • 2 years ago

The Hogwarts Great Hall environment is now live on HBO Max for Apple Vision Pro. Big thanks to my team for bringing it to life — a ton of craft and care went into this one. Can’t wait for you to step inside.

The Hogwarts Great Hall environment is now live on HBO Max for Apple Vision Pro. Big thanks to my team for bringing it to life — a ton of craft and care went into this one. Can’t wait for you to step inside.

ADAM BADΞR ᯅ

34,823 views • 1 year ago

Just tested Runway Aleph 2.0 and this blew my mind a bit lol I saw a video like this when Aleph first released and with the new Aleph 2.0 I wanted to create my own version. You basically select a single frame from the video, apply a change to that specific frame using Nano Banana Pro, GPT Image 2 or Gen-4 and based on the newly generated image Aleph propagates the change through the entire shot. It's really cool seeing the whole sequence adapt from just one edited frame

Just tested Runway Aleph 2.0 and this blew my mind a bit lol I saw a video like this when Aleph first released and with the new Aleph 2.0 I wanted to create my own version. You basically select a single frame from the video, apply a change to that specific frame using Nano Banana Pro, GPT Image 2 or Gen-4 and based on the newly generated image Aleph propagates the change through the entire shot. It's really cool seeing the whole sequence adapt from just one edited frame

Ege

88,298 views • 3 months ago

🇧🇷 APPLE VISON PRO ADOPTED BY BRAZILIAN SURGEON For the first time in Brazil, a surgeon successfully used an Apple Vision Pro headset to optimize a surgical procedure. Surgeon, Bruno Gobbato: "Shoulder arthroscopy surgery uses a camera inside the joint and surgeons perform it by looking directly at a screen. With this device, I was able to see the image on the size of a movie screen with high resolution, as well as being able to see the patient's exams and 3D models in real-time." Experts expect the device to be used routinely in Brazilian healthcare soon. Source: MacMagazine

🇧🇷 APPLE VISON PRO ADOPTED BY BRAZILIAN SURGEON For the first time in Brazil, a surgeon successfully used an Apple Vision Pro headset to optimize a surgical procedure. Surgeon, Bruno Gobbato: "Shoulder arthroscopy surgery uses a camera inside the joint and surgeons perform it by looking directly at a screen. With this device, I was able to see the image on the size of a movie screen with high resolution, as well as being able to see the patient's exams and 3D models in real-time." Experts expect the device to be used routinely in Brazilian healthcare soon. Source: MacMagazine

Mario Nawfal

4,451,573 views • 2 years ago

Want to create an avatar from a single image? FlexAvatar is a transformer model that creates full 360°, high-quality, and expressive 3D head avatar from just a single portrait image in minutes. Real-time Demo: FlexAvatar's lightweight architecture allows both animation and rendering in real-time, enabling interactive user experiences. To create a new 3D head avatar, only one image is required, e.g., from a webcam. The final avatar is ready after 2 minutes. Architecture: Under the hood, FlexAvatar adopts a transformer-based encoder-decoder design. The encoder maps the input image onto a latent avatar space, while the decoder produces 3D Gaussian attribute maps by incorporating the animation signal via cross-attention. The model learns all facial animations directly from the data without relying on pre-built 3D face models. This equips the avatars with realistic facial expressions. The internal avatar latent space can be conveniently used to integrate additional observations of a person via fitting. This enables use-cases where more than one image of a person is available, e.g., from a phone scan of the person. We train jointly on 2D monocular videos and multi-view data. However, in monocular videos, the animation signal leaks the target viewpoint, causing the model to produce incomplete 3D heads. We call this phenomenon entanglement of driving signal and target viewpoint. To prevent entanglement, we introduce bias sinks. These are learnable tokens that indicate whether a training sample stems from a monocular or a multi-view dataset. During training, the model learns to produce incomplete 3D heads only when the monocular token is present. During inference, FlexAvatar then always uses the multi-view token for which the model has learned to produce complete 3D heads. This simple design allows to combine the generalizability from monocular data with the quality of multi-view data. FlexAvatar summary: - Input: Single-image, phone scan, or monocular video - Output: Full 360° head avatar - Expressive animations - Real-time rendering and animation - Generalization to any portrait - Create a new avatar in 2 minutes - Use bias sinks to combine 2D and 3D data 🏠 🌍 🎥 Great work by Tobias Kirschstein and Simon Giebenhain!

Matthias Niessner

96,238 views • 8 months ago