A few months ago, Kyle McDonald sent me a message with a simple idea: capture my workspace as a Gaussian Splat, put a monitor back into that same space, track the viewer and make the monitor disappear. It’s been on my to-do list ever since... I finally built a... POC… Working name: Vitre. It turns a screen into a window onto a 3D capture of the space it is actually in. My office space was captured as a 3D Gaussian Splat and registered back into the physical space. A camera tracks the viewer’s position, and the scene is rendered from that point of view in real time. Move around the screen and the perspective changes with you, as if you were looking through a window. Still a POC. The calibration of the whole system needs to be more precise, and there’s still some noticeable latency between the tracking and the render. #3DGS #GaussianSplatting #ComputerVisionshow more

@kcimc woah now this is cool. could easily see this being placed as a portal to to view historical sites through time, or placed inside a home as the future of family photos

@kcimc Absolutely… I think this could be great in a museum or public setting. That’s part of why I wanted to put this POC together. The other part was to better understand all the steps required to properly align and register a system like this.

@kcimc meanwhile the dude in the other dimension

@kcimc So, mirrors from Prey?

@kcimc 如果有两个的话你就可以把它做成眼镜了

@kcimc but I did that already 😉...

@kcimc You're so amazing, bro

@kcimc Cool idea! Wondering how far you can move your head before the edges of the monitor start disagreeing with the splat.

@kcimc The real limitation is the tracking.

@kcimc Makes sense. Is it latency or jitter that breaks the illusion first?

@kcimc Both are illusion killers... But I’d say the scene isn’t jittering that much. Latency is more of an issue right now, but I’m working on it.

@kcimc The white balance is bugging me

@kcimc Yes, you’re right. There’s definitely some of that. The room was also lit quite differently when I captured it... you can even see a patch of light on one of the walls. But this is something I want to fix in the next iteration.

@kcimc super cool

@kcimc Thanks 🙌😶‍🌫️

@kcimc wow, you could make a real cool digital terrarium with that. put a little guy in there

@kcimc Cool tech too

@kcimc I smell military defense tech in your future 💪🏽

@kcimc Stargate quantum mirror! AR some scenes into it, people looking back at you through the mirror