I'm excited to share our new work Align3R that estimates camera poses and consistent depth maps from a monocular video of a dynamic scene. Project page: Code: Paper:

Our work is motivated by aligning estimated single-view depth maps with DUSt3R. To achieve this, we fine-tuned DUSt3R on 3D scenes with additional monocular depth maps as inputs.

There is another excellent work MonST3R ( that also finetunes DUSt3R on dynamic scenes. We have borrowed the idea of flow losses from MonST3R. We thank the authors for releasing their excellent work.

amazing work

isn't there something to fill in the white gaps?

It is possible to do this with some diffusion models like CAT4D ( This problem is not well-studied yet.

Wondering, if the camera's movement only includes rotation but not translation. what are the results will be.

Thank you! It can work even if the camera is static so I assume that it can handle rotated cameras without camera motion because the predicted 3D point maps provide some cues to solve pure rotations.

So I need to upload a every frame? Not sure how it works.

Yes. The demo only infers a few frames due to the limited GPU resources. Using the GitHub codes will be more convenient if we want to predict a whole video.