Want to obtain time-consistent dynamic mesh from monocular videos? Introducing: Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Monocular Videos We reconstruct meshes with flexible topology change and build the corresp. across meshes. 🧵(1/n)

We also apply our method on real-world captures. Our method is able to produce satisfying meshes given dynamic monocular input videos. 🧵(2/n)

With mesh-to-mesh correspondence, we support downstream applications such as texture editing on dynamic objects. 🧵(3/n)

In our method, we deform canonical 3D Gaussian points into different times and use the differentiable poisson solver and marching cubes to turn the oriented point cloud into the mesh. Because of this, we can reconstruct the mesh with flexible topology change. 🧵(4/n)

To acquire the mesh-to-mesh corresp., we decompose it into two separate corresps. that are easier to acquire: 1. the mesh-to-point corresp. achieved by Gaussian-Mesh Anchoring and 2. the points-to-canonical-point corresp. maintained by the Cycle-Consistent Deformation. 🧵(5/n)

ArXiv: Project page: GitHub: 🧵(6/n)

Nice results, I am wondering the training time, how long does it take for training for like 10 seconds video with 20 fps?

Hi Hongwei, thanks for asking! Currently it takes ~90min to train 200 frames

It looks awsome...great work!

How to compute the cycle consistency loss? G and G' don't have 1-1 correspondence (because of density control, avg&create), how to define F_f(G, t) = F_d(G', t) ? Also F should take (x, t) or their positional encoding as input, what does F(G, t) mean?

Hi Youjia, thanks for your question! F(G, t) means the same thing as F(x, t). G’ should be the form of G under the deformed space, so they have 1-1 corresp.

Oh, there are two G' in the paper. The first one is the G under the deformed space, the second one is the G under the deformed space and anchored by averaging properties & creating new Gaussian at face centroid.

Great work! Very high quality!

Awesome!

Did you work for crucible labs?

awesome work. Will you plan to release the raw iphone captured data and the script for generating the data? I tested the referred code but failed