Introducing CUT3R! An online 3D reasoning framework for many 3D tasks directly from just RGB. For static or dynamic scenes. Video or image collections, all in one!

Key insight is to store 3D information in a latent persistent state that gets continuously updated as images come in. This enables it to continuously refine its predictions as it sees more images!

From the state you can also predict unseen parts. This also improves as you see more. We have a lot of results on the project site: check it out!

Thanks to the amazing team @Ify_Zhang  @holynski_, Alexei Efros, and @akanazawa! Code and model will be released soon!

Customise the colours yourself or simply connect the Joy-Cons ! Head to ➡️ for more details about #AAAClock ! #indiedev #indiegame #IndieGameDev #nintendo #OLED #switchOLED

If you have multiple cameras w/ accurate timestamps does this work still? Seems like it would? Excited to try it out either way!

CUT3R looks like a breakthrough for 3D reasoning! Being able to handle both static and dynamic scenes is incredible. What’s your favorite use case so far?

Tagging @accursedfarms wrt magic software

CUT3R looks like a game-changer for 3D reasoning tasks! Can you share some examples of how it's been used in real-world applications?

@berkeley_ai Fantastic work! I'd love to get pointed to the paper to learn more

Very amazing work! If the input camera is dynamic and contains dynamic objects in the scene, will the reconstructed scene retain the information from past frames, like Dynamic SLAM? I checked out the dynamic demo on the website, but they seem to focus on current frame mainly?