Volumetric Capture is Hard There is no magic solution. Anyone interested in spatial R&D can get access to some awesome datasets online to test with, which can be especially interesting when processed with 3DGS and played back as 4DGS. However you need a ton of compute power. (multiple 3080,... 3090+, RTX5000+ etc) The Necessity of Multiple Cameras for Volumetric Capture Consider, for instance, this very useful fixed rig dataset comprising 20 camera streams provided by Facebook Research Or this datasets comprising 40 camera streams provided by Google The attached video demonstrates FB's RAW 3DGS data played back in real-time in our custom Sibr viewer. It's noticable how quickly the quality can breakdown out of the camera group frustrum. Even at this quality it takes a lot of time and energy to process and playback. It's not simple. Let alone the expertise required by the original research team who captured the data. Doing this with 20 cameras is hard. 150+ cameras even harder. Now imagine trying to achieve it with a single camera phone to playback on a Meta Quest or Spatial headset. #volumetriccaptureishard #camerasdontseearoundcorners #spatialshow more

It's surprising how many people would have you believe that cameras can see around corners :) "We reconstructed this entire cathedral from just one image” There’s a lot of selective camera angles and exaggerated claims circulating currently. The work you're doing is setting the bar. I wouldn’t concern yourself with the misleading information out there; ultimately, the quality of your work and the actual complexity of 360 degree 4D video will become apparent quite quickly once clients start asking for it.

Thanks that's most appreciated. The missinformation does tend to muddy the waters though, which makes it a little harder to navigate becuase of false expectations.

The distortion could make a really cool movie effect about looking through time, kinda like interstellar

Something like a 4DGS in Soccer or UFC should be amazing. Preinstalled cameras recording from different angles, pit dimensions set and don’t require rendering. Only 22 players + ref. Could work?

It's been done for Football by Canon and other companies in the past. Also it's worth a google for the Intel stage (which shuttered). The results are OK. UFC would be cool for sure. You would still need a ton of cameras. 150-200+ it's a multi-million £ project.

Yeah, very hard not to overfit to the provided camera set, even if there's a lot Lots of ideas in research on how to regularize these 3D representations to work with sparse input, but these are mostly heuristics and hard to configure properly. This is hard. I love it

It would be useful to separate the subject somehow from the rest of the scene and if nothing changes in the background keep the background the same until something changes. With reflective surfaces it would be much harder of course.

That's the research we are currently doing. We were just demonstrating real-world capture results. Capturing in situ is hard.

Post it on

Streaming holodeck