Gaussian Splatting in VR, all code written by me from scratch on DX12 + OpenVR.
@charshenton1046 位关注者
121,858 次观看
10 条评论
评论来自 X (Twitter) 上的原帖

@charshenton
Now I've tried this I have a LOT of thoughts to share. 1) I was afraid the approximate affine transform of the gaussians into 2d would not be stereoscopically consistent. I am no longer concerned about that. By the time you get close enough you're out of stereo overlap.
30 个赞

@charshenton
2) I am thinking this is the future of geometry for VR. Why? - It is inherently anti-aliased, which is a big deal for VR - Its performance scales sublinearly with resolution - Geometric density in VR is more compelling than surface texture density
62 个赞

@charshenton
3) Even up close where the "illusion" of photorealism is broken, this is still really visually pleasant. It's like looking at brush strokes on a paint, vs. triangles when it's clipping plane city.
37 个赞

@charshenton
4) Photogrammetry / Novel view synthesis is not the most interesting use case, I want to be able to model these directly. Absolutely no reason you can't just replace SH colour with regular PBR material properties.
33 个赞

@charshenton
5) Speaking of Spherical Harmonics. For their massive data cost they don't add much. Bet you didn't even notice I disabled them for this video ;) (Not for perf reasons, they are by far one of the cheaper parts of the pipeline).
26 个赞

@charshenton
6) On aliasing, the fact we are "shading" per vertex let's us do interesting things like filtering the shaded geometry itself. In VR this could be a good way to combat specular aliasing, since we are always blending + averaging when using gaussian splats.
19 个赞

@charshenton
7) This performance is still unacceptably low. Let's put this in perspective, this is ~5M source primitives, and we're not doing any per vertex or pixel "shading". Not doing any sort of direct or indirect illumination.
25 个赞

@charshenton
8) So way slower than triangles. I think for this to be relevant for games, the geometry pipeline needs to be about 5-10x faster, and we need a way to scale to Billions of source primitives. Time to get working on that I guess.
43 个赞

@ImNeubert
Amazing work
3 个赞

@charshenton
Thanks!