"More Doggo than Doggo" Motion scanned @ 60fps. Playing back @ 20fps. Meet our inhouse tech Doggo "Rolo". Since July we've been redesigning our scanning pipeline to work with the new 3D Gaussian Splatting for Real-Time Radiance Field Rendering method from Inria. GS has reignited our passion for 3D/4D... scanning. Thank you to the team at Inria IR's AeonX capture system has been designed to capture "Spatial" memories for the future. We've built on a custom version of Inria's Sibr viewer to handle the playback of our multiple processed 4D frames. Our next goal is to optimize the player to be multi-threaded to improve performance over 60fps. GS has reignited our passion for 3D/4D scanning. Thank you to the team at Inria More content to come. #guassiansplatting #ir #aeonx #4d #motionscanning #inria #ximea #idatronic #spatial #youth83show more

This is one more impressive than the relighting!

Thanks Jonathan. Just to confirm what we're displaying on the RGB post and White light post are baked lighting scenes. However we have the hardware to be able to capture all the information for re-lighting using machine learning. We're just not there yet on the software side.

Awesome results! you are using multiple cameras, right?

Thanks & yes.

Daamn @skalskip92 you seeing this? 3DGS is now being used for extrapolating multiple angles of recording, not only a frozen moment in time 😻 #thursdai

Wow, super impressive! Curious about file size 🤔. I guess all those nerf/3dGS are lighter weight than traditional formats of this kind, but heavy on the realtime processing?

Compared to Nvidia INGP they are larger. They can average between 20mb to 100mb (cropped) per frame. Uncropped they are can be upto 500mb. We plan to work on frame compression soon.

Wow, very impressive results! It's always exciting to see what new ideas you've been working on, IR. In terms of workload, how was this compared to capturing a 4D scan using the photogrammetric/photometric pipeline? Hopefully this approach is more efficient!

Thanks Classy Dog! It's much easier in that it's just content to play back and visualize. Record, process & playback. We're not doing re-lighting or anything here we're just switching between different captured lighting states that are baked. But we can record all the information required for relighting if clients require. Processing re-lighting is something we want to investigate for the future for machine learning purposes.

This looks so good, especially the eyes. When do you think Guassian splitting workflow will be available to the general public through apps like @LumaLabsAI ?

@maxym_me @LumaLabsAI I'm not sure. They have a lot of invetment funding so anything is possible.