Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Introducing MegaSaM! 🎥 Accurate, fast, & robust structure + camera estimation from casual monocular videos of dynamic scenes! MegaSaM outputs camera parameters and consistent video depth, scaling to long videos with unconstrained camera paths and complex scene dynamics!

Zhengqi Li

1,586 subscribers

56,923 views • 1 year ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

8 Comments

Zhengqi Li1 year ago

🔗See the gallery of results, and try our interactive viewer at 📄Read more: Big thanks to the amazing team! @holynski_, @forrestercole , @QianqianWang5, @jin_linyi, @vickie_ye_, @akanazawa, @Jimantha and Richard Tucker

Soumith Chintala1 year ago

when will the code be released? would love to try it

Angela Dai1 year ago

looks super cool :)

Bilawal Sidhu1 year ago

impressive -- results looks much better than monst3r too -- congrats!

Taro TheTechie1 year ago

Impressive! I'm curious to see how MegaSaM handles complex scenes with multiple moving objects. Can it maintain accuracy in those scenarios?

Nigel Higgs1 year ago

jesus christ this field is moving fast. The weights availble?

Jonathan Sauder1 year ago

Very impressive results, congratulations! Looking forward to the code/checkpoint release!

Ang Cao1 year ago

So cool

Related Videos

I'm excited to share our new work Align3R that estimates camera poses and consistent depth maps from a monocular video of a dynamic scene. Project page: Code: Paper:

I'm excited to share our new work Align3R that estimates camera poses and consistent depth maps from a monocular video of a dynamic scene. Project page: Code: Paper:

Yuan Liu

56,547 views • 1 year ago

"DVD: Dynamic Video Depth" TL;DR: Recovers temporally consistent depth from monocular videos using diffusion priors + geometric constraints, handling dynamic scenes and motion robustly.

"DVD: Dynamic Video Depth" TL;DR: Recovers temporally consistent depth from monocular videos using diffusion priors + geometric constraints, handling dynamic scenes and motion robustly.

Alexandre Morgand

11,804 views • 3 months ago

LongSplat: Robust Unposed 3D Gaussian Splatting for Casual Long Videos Contributions: • An incremental joint optimization approach for simultaneous camera pose and 3DGS reconstruction, reducing local minima and ensuring global consistency. • A robust pose estimation module leveraging learned 3D priors for accurate camera pose estimation. • An adaptive Octree Anchor Formation strategy that significantly reduces memory usage while preserving reconstruction quality.

LongSplat: Robust Unposed 3D Gaussian Splatting for Casual Long Videos Contributions: • An incremental joint optimization approach for simultaneous camera pose and 3DGS reconstruction, reducing local minima and ensuring global consistency. • A robust pose estimation module leveraging learned 3D priors for accurate camera pose estimation. • An adaptive Octree Anchor Formation strategy that significantly reduces memory usage while preserving reconstruction quality.

MrNeRF

35,969 views • 10 months ago

Introducing Stable Virtual Camera: This multi-view diffusion model transforms 2D images into immersive 3D videos with realistic depth and perspective—without complex reconstruction or scene-specific optimization.

Introducing Stable Virtual Camera: This multi-view diffusion model transforms 2D images into immersive 3D videos with realistic depth and perspective—without complex reconstruction or scene-specific optimization.

Stability AI

197,252 views • 1 year ago

SceneScape: Text-Driven Consistent Scene Generation abs: project page: text-driven perpetual view generation -- synthesizing long videos of arbitrary scenes solely from an input text describing the scene and camera poses

SceneScape: Text-Driven Consistent Scene Generation abs: project page: text-driven perpetual view generation -- synthesizing long videos of arbitrary scenes solely from an input text describing the scene and camera poses

AK

73,258 views • 3 years ago

ViPE: Video Pose Engine for 3D Geometric Perception Contributions: • A robust and efficient framework, ViPE, for estimating camera parameters and dense depth from diverse, in-the-wild videos. • A system design that integrates the strengths of classical SLAM (efficiency, scalability) and learned models (robustness), with key improvements in efficiency, dynamic object handling, and depth quality over prior work. • A large-scale dataset of annotated videos, created using ViPE, to facilitate future research in 3D computer vision.

ViPE: Video Pose Engine for 3D Geometric Perception Contributions: • A robust and efficient framework, ViPE, for estimating camera parameters and dense depth from diverse, in-the-wild videos. • A system design that integrates the strengths of classical SLAM (efficiency, scalability) and learned models (robustness), with key improvements in efficiency, dynamic object handling, and depth quality over prior work. • A large-scale dataset of annotated videos, created using ViPE, to facilitate future research in 3D computer vision.

MrNeRF

42,534 views • 10 months ago

Thrilled to introduce Video Depth Anything to support Depth Estimation for super-long videos (over 5 minutes). 👉It enjoys all the benefits of #DepthAnything: high-quality, fast, robust, etc. Proj Page:

Thrilled to introduce Video Depth Anything to support Depth Estimation for super-long videos (over 5 minutes). 👉It enjoys all the benefits of #DepthAnything: high-quality, fast, robust, etc. Proj Page:

Bingyi Kang

25,464 views • 1 year ago

Exciting news! MegaSAM code is out🔥 & the updated Shape of Motion results with MegaSAM are really impressive! A year ago I didn't think we could make any progress on these videos: Huge congrats to everyone involved and the community 🎉

Exciting news! MegaSAM code is out🔥 & the updated Shape of Motion results with MegaSAM are really impressive! A year ago I didn't think we could make any progress on these videos: Huge congrats to everyone involved and the community 🎉

Angjoo Kanazawa

95,476 views • 1 year ago

Creating a consistent scene and character. Seedream is able to edit camera angles from a single reference image. This gives you more control over your video outputs with start and end frames.

Creating a consistent scene and character. Seedream is able to edit camera angles from a single reference image. This gives you more control over your video outputs with start and end frames.

Jerrod Lew

26,972 views • 9 months ago

CamCloneMaster: replicating camera movements from reference videos without needing camera parameters; based on Wan2.1-1.3B; built using a large-scale Camera Clone Dataset. Solid results.

CamCloneMaster: replicating camera movements from reference videos without needing camera parameters; based on Wan2.1-1.3B; built using a large-scale Camera Clone Dataset. Solid results.

Wildminder

16,829 views • 7 months ago

Introducing PixVerse 5.5 Now with 10-second videos, native audio generation, dynamic multi-clip camera work, and smarter prompt optimization Available now on Freepik

Introducing PixVerse 5.5 Now with 10-second videos, native audio generation, dynamic multi-clip camera work, and smarter prompt optimization Available now on Freepik

Magnific

10,433 views • 6 months ago

Check out our fun work on robust view synthesis! 🤩 Using a casual video as input, our method jointly 1⃣estimates accurate camera poses and 2⃣reconstructs multiple local radiance fields for a large scene. Project🕸️:

Check out our fun work on robust view synthesis! 🤩 Using a casual video as input, our method jointly 1⃣estimates accurate camera poses and 2⃣reconstructs multiple local radiance fields for a large scene. Project🕸️:

Jia-Bin Huang

35,526 views • 3 years ago

🚨Dynamic Camera Tutorial🚨 Struggling to get dynamic camera movements in your AI videos? We cracked the code! 🚀 Here's how we craft dynamic videos with stunning results using a Master Prompt Template for ChatGPT (or any other LLM) that incorporates dynamic camera movements! Creative process + examples 👇 Sound On 🔉

🚨Dynamic Camera Tutorial🚨 Struggling to get dynamic camera movements in your AI videos? We cracked the code! 🚀 Here's how we craft dynamic videos with stunning results using a Master Prompt Template for ChatGPT (or any other LLM) that incorporates dynamic camera movements! Creative process + examples 👇 Sound On 🔉

Nim

36,993 views • 1 year ago

🎥✨Camera Motion is here in Dream Machine 1.6! Effortlessly direct text-to-video and image-to-video scenes with simple commands! Just type in ‘camera’ to unlock camera motions! Imagine in #LumaDreamMachine 🎬🚀

🎥✨Camera Motion is here in Dream Machine 1.6! Effortlessly direct text-to-video and image-to-video scenes with simple commands! Just type in ‘camera’ to unlock camera motions! Imagine in #LumaDreamMachine 🎬🚀

Luma

260,904 views • 1 year ago

New Video! Excited to share our work on fast view synthesis of casual videos. 🤩 high-quality dynamic view synthesis 🤩 applicable for a wide range of casual videos 🤩 fast training and rendering

New Video! Excited to share our work on fast view synthesis of casual videos. 🤩 high-quality dynamic view synthesis 🤩 applicable for a wide range of casual videos 🤩 fast training and rendering

Jia-Bin Huang

27,360 views • 2 years ago

Multi-view Reconstruction via SfM-guided Monocular Depth Estimation Contributions: • We propose a novel approach to inject SfM priors into diffusion-based depth estimation, enabling highly accurate and multi-view consistent depth predictions for each viewpoint. • Based on the proposed depth estimator, we design a new multi-view 3D geometry reconstruction framework and process some synthetic datasets to facilitate training. • We evaluate our method on diverse real-world scene data, including objects, indoor environments, streetscapes, and aerial scenes, demonstrating the superior performance and generalization capability of our approach.

Multi-view Reconstruction via SfM-guided Monocular Depth Estimation Contributions: • We propose a novel approach to inject SfM priors into diffusion-based depth estimation, enabling highly accurate and multi-view consistent depth predictions for each viewpoint. • Based on the proposed depth estimator, we design a new multi-view 3D geometry reconstruction framework and process some synthetic datasets to facilitate training. • We evaluate our method on diverse real-world scene data, including objects, indoor environments, streetscapes, and aerial scenes, demonstrating the superior performance and generalization capability of our approach.

MrNeRF

25,651 views • 1 year ago

Excited to share MonST3R! -- a simple way to estimate geometry from unposed video of dynamic scene We achieve competitive results on several downstreams (video depth, camera pose) and believe this is a promising step toward feed-forward 4D reconstruction

Excited to share MonST3R! -- a simple way to estimate geometry from unposed video of dynamic scene We achieve competitive results on several downstreams (video depth, camera pose) and believe this is a promising step toward feed-forward 4D reconstruction

Junyi Zhang

131,541 views • 1 year ago

Some additional scenes from Kyiv. All videos are from the same security camera.

Some additional scenes from Kyiv. All videos are from the same security camera.

AMK Mapping 🇳🇿

11,575 views • 9 months ago

Can video generative models exhibit visuospatial intelligence? 🤔 Introducing Video4Spatial — a video-only framework that tackles spatial tasks. With just video context, our model can: 🔍 Ground objects by planning geometry-consistent paths 📸 Follow camera-pose instructions for scene navigation 🌐 Generalize to long contexts & unseen outdoor scenes A step toward video models as visual-spatial reasoners. Project: arXiv:

Can video generative models exhibit visuospatial intelligence? 🤔 Introducing Video4Spatial — a video-only framework that tackles spatial tasks. With just video context, our model can: 🔍 Ground objects by planning geometry-consistent paths 📸 Follow camera-pose instructions for scene navigation 🌐 Generalize to long contexts & unseen outdoor scenes A step toward video models as visual-spatial reasoners. Project: arXiv:

Xingang Pan

15,902 views • 6 months ago