跳过导航
Gaussian Splatting in VR, all code written by me from scratch on DX12 + OpenVR.
0:00 / 1:00

Gaussian Splatting in VR, all code written by me from scratch on DX12 + OpenVR.

Charlie Shenton

@charshenton1046 位关注者

在 X 上查看
1,033 个赞下载原帖

121,858 次观看

Gaussian Splatting in VR, all code written by me from scratch on DX12 + OpenVR. 2x2016x2240 res, no clever multi-view stuff yet so just running the whole render pipeline once per eye. Left the frame timer on so you can see I've been truthful about my perf :D

10 条评论

评论来自 X (Twitter) 上的原帖

  • Charlie Shenton 的头像

    @charshenton

    Now I've tried this I have a LOT of thoughts to share. 1) I was afraid the approximate affine transform of the gaussians into 2d would not be stereoscopically consistent. I am no longer concerned about that. By the time you get close enough you're out of stereo overlap.

    30 个赞

  • Charlie Shenton 的头像

    @charshenton

    2) I am thinking this is the future of geometry for VR. Why? - It is inherently anti-aliased, which is a big deal for VR - Its performance scales sublinearly with resolution - Geometric density in VR is more compelling than surface texture density

    62 个赞

  • Charlie Shenton 的头像

    @charshenton

    3) Even up close where the "illusion" of photorealism is broken, this is still really visually pleasant. It's like looking at brush strokes on a paint, vs. triangles when it's clipping plane city.

    37 个赞

  • Charlie Shenton 的头像

    @charshenton

    4) Photogrammetry / Novel view synthesis is not the most interesting use case, I want to be able to model these directly. Absolutely no reason you can't just replace SH colour with regular PBR material properties.

    33 个赞

  • Charlie Shenton 的头像

    @charshenton

    5) Speaking of Spherical Harmonics. For their massive data cost they don't add much. Bet you didn't even notice I disabled them for this video ;) (Not for perf reasons, they are by far one of the cheaper parts of the pipeline).

    26 个赞

  • Charlie Shenton 的头像

    @charshenton

    6) On aliasing, the fact we are "shading" per vertex let's us do interesting things like filtering the shaded geometry itself. In VR this could be a good way to combat specular aliasing, since we are always blending + averaging when using gaussian splats.

    19 个赞

  • Charlie Shenton 的头像

    @charshenton

    7) This performance is still unacceptably low. Let's put this in perspective, this is ~5M source primitives, and we're not doing any per vertex or pixel "shading". Not doing any sort of direct or indirect illumination.

    25 个赞

  • Charlie Shenton 的头像

    @charshenton

    8) So way slower than triangles. I think for this to be relevant for games, the geometry pipeline needs to be about 5-10x faster, and we need a way to scale to Billions of source primitives. Time to get working on that I guess.

    43 个赞

  • NeubertIM 📯 的头像

    @ImNeubert

    Amazing work

    3 个赞

  • Charlie Shenton 的头像

    @charshenton

    Thanks!