I've implemented a Gaussian Splatting renderer in VRChat! It was tricky, but turned out to be feasible. I've also made a public gallery with 12 splats you can check out: By the way the renderer is open source!

Now onto the implementation details: the main issue preventing a GS renderer in VRChat was the lack of good sorting, you could do bitonic, sure, but that would be very slow, or precompute the sorting order, but that's gonna take a huge amount of VRAM for large splats.

Turned out you can just implement radix sort in VRC, you don't really need compute for it to work. If you used @d4rkpl4y3r_vr's trick for mipmap prefix sums its actually quite fast You basically compute the prefix sum over digit histograms in a fragment shader - then in a separate pass do a binary search over the prefix sum (which you compute on the fly) to find the sorted element ID (since you cant do scatter, you must search instead)

Of course even if you have the rendering order correct, you still need to draw the splats, and there are a lot of already existing implementations. For VR specifically you need perspective correct gaussian splatting, otherwise it looks ugly closeup.

But even then, since we dont have compute, we must rasterize the splats as quads, which get very slow very fast especially for VR resolutions, with all that overdraw.

So to optimize I tried to estimate maximally tight quads for the gaussians via projecting an ellipsoid onto the screen to find an screen space ellipse. You can do that, but its a lot of math and very numerically unstable. I needed to use emulated doubles for this to work.

The overhead of the [emulated] doubles is relatively tiny compared to the cost of rasterizing the splats, so it was fine. Of course since the ellipse we get is the exact projection it is technically perspective correct, or at least its outline would be, so we can keep the fragment shader relatively simple as in EWA splatting, and it looks fine.

Another minor cursed thing is that VRChat colorspace is linear, while splats are in sRGB, to solve this I convert the entire screen to sRGB before splatting, then to linear after splatting, which is cursed and takes a bit of performance, but it works.

I hope to see you guys using this to make some nice gaussian splat worlds, or integrate it into one in some way. Would be cool!

Scan any documents, convert images into text, PDF files, etc. 👍

I've been waiting on something like this for a while, and immediately uploaded a few of my splats when I saw it. Thank you so much for this- you're a legend! ^^

I hope its a public world? I want to look at more splats 👀