Azad Balabanian's banner

Azad Balabanian

@Azadux • 11,522 subscribers

3D scanning/recon—Product @nianticspatial, formerly @puzzlingplaces, @uploadvr. Perpetuel typos. Opinions my own

Shorts

had my wife scan me. not bad, not great. likely needs a lot more photos

had my wife scan me. not bad, not great. likely needs a lot more photos

60,534 просмотров

Apple’s spatial photo viewer is quite advanced It uses the metadata of the 3D camera (FOV, interaxial distance, focal length) to adjust the projection and match the scale of real life Here, my wife has the right height and roundness (heh) as IRL It looks SO good to look at in AVP (link below)

Apple’s spatial photo viewer is quite advanced It uses the metadata of the 3D camera (FOV, interaxial distance, focal length) to adjust the projection and match the scale of real life Here, my wife has the right height and roundness (heh) as IRL It looks SO good to look at in AVP (link below)

65,612 просмотров

splat troubleshooting guide: If you ever export a splat from NerfStudio that's transparent, let it train for another 1000 steps what's happening is the alphas get reset every 3k steps, so here when i exported at 12k steps, it was right when the alphas reset, hence the transparency. you can adjust the reset frequency with reset_every: int = 3000 this might've been obvious to some, but I was confused by it last night and found no discussions/explanation about it online. so posting it here to help others

splat troubleshooting guide: If you ever export a splat from NerfStudio that's transparent, let it train for another 1000 steps what's happening is the alphas get reset every 3k steps, so here when i exported at 12k steps, it was right when the alphas reset, hence the transparency. you can adjust the reset frequency with reset_every: int = 3000 this might've been obvious to some, but I was confused by it last night and found no discussions/explanation about it online. so posting it here to help others

37,796 просмотров

Some personal news! 📣 Recently, I joined Niantic as the product lead to work on MR apps and Gaussian splatting! Today, we’re launching “Into The Scaniverse”—A way of viewing Scaniverse splats in VR (yours and the ones posted to the map) 📸 It’s available as a webxr app via Quest Browser (for Q3 and Q3S), soon to launch in the Meta Horizon store. Honored to be working with quite a lot of brilliant folks at Niantic 🙌 Link to the webxr app 👇

Some personal news! 📣 Recently, I joined Niantic as the product lead to work on MR apps and Gaussian splatting! Today, we’re launching “Into The Scaniverse”—A way of viewing Scaniverse splats in VR (yours and the ones posted to the map) 📸 It’s available as a webxr app via Quest Browser (for Q3 and Q3S), soon to launch in the Meta Horizon store. Honored to be working with quite a lot of brilliant folks at Niantic 🙌 Link to the webxr app 👇

44,645 просмотров

Found a really cool accessibility setting in visionOS that exposes your eye tracking directly It’s really trippy to see your own saccades, especially when reading lines of texts.

Found a really cool accessibility setting in visionOS that exposes your eye tracking directly It’s really trippy to see your own saccades, especially when reading lines of texts.

41,023 просмотров

Gaussian splatting is so good at capturing context. For me, it has changed my approach to volumetric photography. For me with photogrammetry, it was very much either object-based, space based, or facade-based, always with the subject’s bounds being very clearly defined during capture But with splats, I can think more frustum-based, composing a scene with a foreground, background, and subject, and shooting it. It gives me back the freedom I had with traditional photography

Gaussian splatting is so good at capturing context. For me, it has changed my approach to volumetric photography. For me with photogrammetry, it was very much either object-based, space based, or facade-based, always with the subject’s bounds being very clearly defined during capture But with splats, I can think more frustum-based, composing a scene with a foreground, background, and subject, and shooting it. It gives me back the freedom I had with traditional photography

12,909 просмотров

Scan -> viewing in VR in less than 5 minutes It is pretty neat to have a Gaussian Splatting capturing platform (Scaniverse) + cloud storage + VR viewer (Into the Scaniverse) under one Niantic account

Scan -> viewing in VR in less than 5 minutes It is pretty neat to have a Gaussian Splatting capturing platform (Scaniverse) + cloud storage + VR viewer (Into the Scaniverse) under one Niantic account

11,639 просмотров

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

For people who don’t get the 2D gaming on virtual monitors in VR use case: HMDs have the potential to be the ultimate monitor for any content. Any aspect ratio, any resolution, any size, it’s all “native” in vr I get to play my game on a massive screen while I have spatial awareness, see my wife, take a sip of coffee, and have depth correct rendering content in my room. If and when I choose to, I can “turn off” my surrounds and go into a virtual environment. It’s MR by default, VR by choice Nobody has really taken this use case all that seriously and optimized the hardware/software around it. Now, it seems like Valve is. Also some context: I’m streaming SW Jedi Fallen Order at 4k 90fps my PC to my Vision Pro, and playing the audio thru my surround sound speakers. It’s the best of ALL worlds

For people who don’t get the 2D gaming on virtual monitors in VR use case: HMDs have the potential to be the ultimate monitor for any content. Any aspect ratio, any resolution, any size, it’s all “native” in vr I get to play my game on a massive screen while I have spatial awareness, see my wife, take a sip of coffee, and have depth correct rendering content in my room. If and when I choose to, I can “turn off” my surrounds and go into a virtual environment. It’s MR by default, VR by choice Nobody has really taken this use case all that seriously and optimized the hardware/software around it. Now, it seems like Valve is. Also some context: I’m streaming SW Jedi Fallen Order at 4k 90fps my PC to my Vision Pro, and playing the audio thru my surround sound speakers. It’s the best of ALL worlds

Azad Balabanian

616,693 просмотров • 6 месяцев назад

accidentally loaded a 2 splats together (SF + church in Ani) and the result is kinda cool

accidentally loaded a 2 splats together (SF + church in Ani) and the result is kinda cool

Azad Balabanian

142,246 просмотров • 2 месяцев назад

a few years ago, I painstaikingly scanned my old 4Runner right before selling it. Today I reprocessed the data as a gaussian splat It was a great little offroader. I'll visit it in VR some day and feel like I'm sitting back in the driver seat.

a few years ago, I painstaikingly scanned my old 4Runner right before selling it. Today I reprocessed the data as a gaussian splat It was a great little offroader. I'll visit it in VR some day and feel like I'm sitting back in the driver seat.

Azad Balabanian

995,882 просмотров • 1 год назад

drone scanned 3 gaussian splats in the South Tyrol in Italy uploaded on #supersplat (link below)

drone scanned 3 gaussian splats in the South Tyrol in Italy uploaded on #supersplat (link below)

Azad Balabanian

644,553 просмотров • 1 год назад

this looks insane

this looks insane

Azad Balabanian

364,339 просмотров • 1 год назад

capitola drone splat 10 min of scanning 1 min of alignment 10 min of splat training

capitola drone splat 10 min of scanning 1 min of alignment 10 min of splat training

Azad Balabanian ᯅ

141,528 просмотров • 8 месяцев назад

Woah, Gaussian splats in mixed reality look so good! Tech stack: lichtfeld studio for splattering training, virtual desktop for PC VR streaming and passthrough MR using a virtual green screen, gracia for VR splat rendering using the green screen background feature

Woah, Gaussian splats in mixed reality look so good! Tech stack: lichtfeld studio for splattering training, virtual desktop for PC VR streaming and passthrough MR using a virtual green screen, gracia for VR splat rendering using the green screen background feature

Azad Balabanian

38,020 просмотров • 2 месяцев назад

life at home a 360 gaussian splat, shot with Insta360 X5 and trained in Jawset PostShot

life at home a 360 gaussian splat, shot with Insta360 X5 and trained in Jawset PostShot

Azad Balabanian

92,251 просмотров • 9 месяцев назад

a 360 camera Gaussian Splat of my wife in a serene, foggy afternoon in Golden Gate Park Captured with an Insta360 X5, aligned and trained in PostShot, thanks to a tip Olli Huttunen. Supersplat link below 👇 This might be the first 360 splat I've made that I'm happy with. It's clear that the accuracy of the image alignment is paramount, and I haven't been getting great results from RealityScan aligning cropped frames from 360 stitches. Here, I aliged and trained the frames directly in PostShot. Alignment took a few hours but the accuracy is great.

a 360 camera Gaussian Splat of my wife in a serene, foggy afternoon in Golden Gate Park Captured with an Insta360 X5, aligned and trained in PostShot, thanks to a tip Olli Huttunen. Supersplat link below 👇 This might be the first 360 splat I've made that I'm happy with. It's clear that the accuracy of the image alignment is paramount, and I haven't been getting great results from RealityScan aligning cropped frames from 360 stitches. Here, I aliged and trained the frames directly in PostShot. Alignment took a few hours but the accuracy is great.

Azad Balabanian ᯅ

89,990 просмотров • 9 месяцев назад

self portait gaussian splat I captured this by myself with a single camera. I mounted the cam, sat on an office chair, and swiveled around a full 360 while recording a video (check next tweet). Used RealityScan's AI masking to mask myself from the surroundings (for sfm to work), then used MrNeRF's LichtFeldStudio to splat! alignment is faaar from perfect (notice the ghosting/doubling), but not a bad result for such a simple workflow! and of course, Codex and I wrote a end-to-end video to splat script to repeate this as many times as I'd like :) view on supersplat:

self portait gaussian splat I captured this by myself with a single camera. I mounted the cam, sat on an office chair, and swiveled around a full 360 while recording a video (check next tweet). Used RealityScan's AI masking to mask myself from the surroundings (for sfm to work), then used MrNeRF's LichtFeldStudio to splat! alignment is faaar from perfect (notice the ghosting/doubling), but not a bad result for such a simple workflow! and of course, Codex and I wrote a end-to-end video to splat script to repeate this as many times as I'd like :) view on supersplat:

Azad Balabanian

32,489 просмотров • 3 месяцев назад

Vision Pro vs Quest 3 Passthrough IMO, the biggest reason people are underwhelmed with Vision Pro’s Passthrough is likely due to the muted/neutral tone mapping and contrast I think the Vision Pro’s cameras have higher dynamic range but their tone mapping is really muted.

Vision Pro vs Quest 3 Passthrough IMO, the biggest reason people are underwhelmed with Vision Pro’s Passthrough is likely due to the muted/neutral tone mapping and contrast I think the Vision Pro’s cameras have higher dynamic range but their tone mapping is really muted.

Azad Balabanian

163,546 просмотров • 2 лет назад

I can definitively say that this 2010 video from Oliver Kreylos was a core part in awakening my love for volumetric photography

I can definitively say that this 2010 video from Oliver Kreylos was a core part in awakening my love for volumetric photography

Azad Balabanian ᯅ

46,924 просмотров • 9 месяцев назад

this might be the most detailed splat i've made yet - LichtFeld Studio MRNF 115k steps Church of the Holy Apostles - Ani, Turkey (2022)

this might be the most detailed splat i've made yet - LichtFeld Studio MRNF 115k steps Church of the Holy Apostles - Ani, Turkey (2022)

Azad Balabanian

15,433 просмотров • 2 месяцев назад

Funny how neither Quest’s or Vision Pro’s hand tracking have anywhere near the fidelity or performance of Leap Motion even after 13 years

Funny how neither Quest’s or Vision Pro’s hand tracking have anywhere near the fidelity or performance of Leap Motion even after 13 years

Azad Balabanian ᯅ

56,382 просмотров • 1 год назад

Overlooked aspect of Steam Frame: SteamOS on ARM and its FEX emulation layer means you’ll be playing PC games on ANY Android/snapdragon device Technically this is already possible with projects like Winlator and GameHub but this would be an “official” and optimized way of doing so

Overlooked aspect of Steam Frame: SteamOS on ARM and its FEX emulation layer means you’ll be playing PC games on ANY Android/snapdragon device Technically this is already possible with projects like Winlator and GameHub but this would be an “official” and optimized way of doing so

Azad Balabanian

29,040 просмотров • 6 месяцев назад

Star Wars squadrons’ cockpit looks so stunning in super sampled VR The lighting, the detail. It’s all so good!

Star Wars squadrons’ cockpit looks so stunning in super sampled VR The lighting, the detail. It’s all so good!

Azad Balabanian ᯅ

54,226 просмотров • 1 год назад

Using my Vision Pro to test out a photo framing before I order a print

Using my Vision Pro to test out a photo framing before I order a print

Azad Balabanian

36,387 просмотров • 1 год назад

Ngl, I’m pretty disappointed by the fact that most of the most by Apple in the AVP Spatial Gallery are AI converted photos And the rest are iPhone shot spatial videos. Nothing even close to professional grade To the untrained eye, AI converted photos “pass” but honestly if I show you the difference between real 3D and converters 3D and you recognize the insane amount of detail that’s missing

Ngl, I’m pretty disappointed by the fact that most of the most by Apple in the AVP Spatial Gallery are AI converted photos And the rest are iPhone shot spatial videos. Nothing even close to professional grade To the untrained eye, AI converted photos “pass” but honestly if I show you the difference between real 3D and converters 3D and you recognize the insane amount of detail that’s missing

Azad Balabanian ᯅ

31,924 просмотров • 1 год назад

Lateral2Spatial is coming along nicely—here’s a preview of how it works - Shoot lateral video (on a drone or phone) - process in app - export out SBS and Spatial Video to watch in Vision Pro (or any other 3D display)! TestFlight email signups below 👇

Lateral2Spatial is coming along nicely—here’s a preview of how it works - Shoot lateral video (on a drone or phone) - process in app - export out SBS and Spatial Video to watch in Vision Pro (or any other 3D display)! TestFlight email signups below 👇

Azad Balabanian

26,246 просмотров • 10 месяцев назад

The new Indiana jones game is best looking game I’ve ever played The full path tracing mode is truly the first time I’ve been awestruck with ray tracing. Every room GI, every material’s speculars, every object on a shelf, every leaf and branch just looks… right! To me, it’s the first time I’ve felt like a game has taken a “next-gen” leap in visuals over the past decade or two honestly. Unlike hellblade II, this game is also really fun and charming, with incredibly well written and acted cutscenes, fantastic sound, and great gameplay. Way better than it has any right to be. Easily my GOTY

The new Indiana jones game is best looking game I’ve ever played The full path tracing mode is truly the first time I’ve been awestruck with ray tracing. Every room GI, every material’s speculars, every object on a shelf, every leaf and branch just looks… right! To me, it’s the first time I’ve felt like a game has taken a “next-gen” leap in visuals over the past decade or two honestly. Unlike hellblade II, this game is also really fun and charming, with incredibly well written and acted cutscenes, fantastic sound, and great gameplay. Way better than it has any right to be. Easily my GOTY

Azad Balabanian

34,477 просмотров • 1 год назад