Loading video...

Video Failed to Load

Go Home

Discover the right 3D Geometric Foundation Model for your task—whether it’s stereo matching, multi-view depth estimation, video depth, pose estimation, semantic understanding, or novel view synthesis. Explore more insights in our #E3DBench #FoundationModel #3D #GaussianSplatting. Project Webpage:

zhiwen_fan_'s profile picture

Zhiwen(Aaron) Fan

1,741 subscribers

29,920 views • 1 year ago •via X (Twitter)

ArtsScience & TechnologyEducation#E3DBench#FoundationModel#3D#GaussianSplatting
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Spent some time this week playing with depth estimation models. Monocular depth estimation has come a long way, but in terms of metric accuracy and geometric consistency, stereo depth still has it beat by a wide margin. DepthPro (mono) vs FoundationStereo (stereo)
0:14

Spent some time this week playing with depth estimation models. Monocular depth estimation has come a long way, but in terms of metric accuracy and geometric consistency, stereo depth still has it beat by a wide margin. DepthPro (mono) vs FoundationStereo (stereo)

FrostyFridge

27,774 views • 1 year ago

I always thought camera pose estimation is necessary for 3D reconstruction, until Zequn and Stephen proved me wrong! Introducing PreF3R, purely feed-forward 3D Gaussian Splatting without any intermediate pose estimation and COLMAP initialization. Video in, 3D Gaussians and novel view rendering out. Key technique: spatial memory network from Spann3R and Gaussian head supervised by pointmap loss plus photometric loss. See more at:
0:19

I always thought camera pose estimation is necessary for 3D reconstruction, until Zequn and Stephen proved me wrong! Introducing PreF3R, purely feed-forward 3D Gaussian Splatting without any intermediate pose estimation and COLMAP initialization. Video in, 3D Gaussians and novel view rendering out. Key technique: spatial memory network from Spann3R and Gaussian head supervised by pointmap loss plus photometric loss. See more at:

Heng Yang

20,160 views • 1 year ago

random model find; might be the best 3D multi-person pose estimation I have ever seen; it's Apache-2.0 Multi-HMR: Regressing Whole-Body Human Meshes for Multiple Persons in a Single Shot; ECCV2024
0:17

random model find; might be the best 3D multi-person pose estimation I have ever seen; it's Apache-2.0 Multi-HMR: Regressing Whole-Body Human Meshes for Multiple Persons in a Single Shot; ECCV2024

SkalskiP

31,150 views • 1 year ago

Learn how to combine a deep learning model for pose estimation to perform a 3D reconstruction using two cameras 📸
0:13

Learn how to combine a deep learning model for pose estimation to perform a 3D reconstruction using two cameras 📸

MATLAB

15,665 views • 1 year ago

Google just revealed an ABSOLUTE depth estimation model 🤯 As opposed to recent depth models (Marigold, PatchFusion) which aim for maximum details, DMD aims to estimate the ABSOLUTE depth (in meters) within the image More details below ⬇️⬇️
0:16

Google just revealed an ABSOLUTE depth estimation model 🤯 As opposed to recent depth models (Marigold, PatchFusion) which aim for maximum details, DMD aims to estimate the ABSOLUTE depth (in meters) within the image More details below ⬇️⬇️

Alex Carlier

199,524 views • 2 years ago

Monocular pose estimation has gotten really good Grab any 2D video and transfer the performance to a 3D character
0:11

Monocular pose estimation has gotten really good Grab any 2D video and transfer the performance to a 3D character

Bilawal Sidhu

27,074 views • 1 year ago

🤔Applying Depth Estimation models directly to videos can result in inconsistency between frames. 💪Well, not anymore. 🔥ChronoDepth is a new approach to video depth estimation that focuses on achieving both accuracy within each frame and consistency across frames. 📜More👇
0:15

🤔Applying Depth Estimation models directly to videos can result in inconsistency between frames. 💪Well, not anymore. 🔥ChronoDepth is a new approach to video depth estimation that focuses on achieving both accuracy within each frame and consistency across frames. 📜More👇

Gradio

19,986 views • 2 years ago

Bai et al., "Positional Encoding Field" Make your RoPE encoding 3D by including a z axis, then manipulate your image by simply manipulating your positional encoding in 3D --> novel view synthesis. Neat idea.
0:19

Bai et al., "Positional Encoding Field" Make your RoPE encoding 3D by including a z axis, then manipulate your image by simply manipulating your positional encoding in 3D --> novel view synthesis. Neat idea.

Kwang Moo Yi

46,855 views • 7 months ago

Video diffusion models are just overqualified depth estimators! Deterministic single-pass depth estimation based on WanV2.1. - SOTA 5.5 AbsRel on ScanNet - data-efficient than baselines; - no temporal flicker + infinite-length estimation w/ zero scale drift.
0:18

Video diffusion models are just overqualified depth estimators! Deterministic single-pass depth estimation based on WanV2.1. - SOTA 5.5 AbsRel on ScanNet - data-efficient than baselines; - no temporal flicker + infinite-length estimation w/ zero scale drift.

Wildminder

49,251 views • 3 months ago

How can we use test-time compute for spatial understanding? 🤔 In InterPose, we propose to repeatedly sample generative video models to help two-view pose estimation and reconstruction, by leveraging the video models' keyframe interpolation abilities. A 🧵... (1/8)
0:17

How can we use test-time compute for spatial understanding? 🤔 In InterPose, we propose to repeatedly sample generative video models to help two-view pose estimation and reconstruction, by leveraging the video models' keyframe interpolation abilities. A 🧵... (1/8)

Ricardo Martin-Brualla

21,685 views • 1 year ago

Gave a robot 3D vision with just a regular camera👁️ Full Tutorial: Deployed #Depth Anything V3​ on NVIDIA Robotics Jetson AGX Orin. It estimates depth from 2D images in real-time—no special sensors needed. Just need #Monocular depth estimation + #TensorRT​ optimization + #ROS2​ integration. 👉 Learn more about reComputer Robotics J5011: #TheAIHardwarePartner
0:18

Gave a robot 3D vision with just a regular camera👁️ Full Tutorial: Deployed #Depth Anything V3​ on NVIDIA Robotics Jetson AGX Orin. It estimates depth from 2D images in real-time—no special sensors needed. Just need #Monocular depth estimation + #TensorRT​ optimization + #ROS2​ integration. 👉 Learn more about reComputer Robotics J5011: #TheAIHardwarePartner

Seeed Studio

16,966 views • 3 months ago

The Stable Video Diffusion model just dropped 🔥 The new model supports: – Text-to-Video – Image-to-Video – 14 or 25 frames at 576 x 1024 – Multi-View Generation – Frame Interpolation – 3D Scene Understanding – Camera Control via LoRA Paper: Code: SVD model: SVD-XT model:
0:10

The Stable Video Diffusion model just dropped 🔥 The new model supports: – Text-to-Video – Image-to-Video – 14 or 25 frames at 576 x 1024 – Multi-View Generation – Frame Interpolation – 3D Scene Understanding – Camera Control via LoRA Paper: Code: SVD model: SVD-XT model:

Dreaming Tulpa 🥓👑

457,688 views • 2 years ago

AnyDepth: Lightweight zero-shot monocular depth estimation; surpasses DPT; - nicely preserves detail.
0:17

AnyDepth: Lightweight zero-shot monocular depth estimation; surpasses DPT; - nicely preserves detail.

Wildminder

18,152 views • 5 months ago

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.
0:13

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

MrNeRF

27,428 views • 1 year ago

hi all, excited to join! i'm building an expressive mini "shoggoth" robot which will eventually be hooked up to gpt4o realtime voice. i'm currently working on the low-level policies, which are trained in a mujoco simulation with RL. to delay working on raw-pixels for now, i trained a pose-estimation model using deeplabcut and triangulate the position in 3d space using the stereo cameras. eventually, i'll use gpt4o's tool calling capabilities to activate several of these policies (closed and open loop) based on the dialog flow! captions: manual actuation of the tentacle / 3d pose estimation / target design
0:29

hi all, excited to join! i'm building an expressive mini "shoggoth" robot which will eventually be hooked up to gpt4o realtime voice. i'm currently working on the low-level policies, which are trained in a mujoco simulation with RL. to delay working on raw-pixels for now, i trained a pose-estimation model using deeplabcut and triangulate the position in 3d space using the stereo cameras. eventually, i'll use gpt4o's tool calling capabilities to activate several of these policies (closed and open loop) based on the dialog flow! captions: manual actuation of the tentacle / 3d pose estimation / target design

Matthieu LC

45,506 views • 1 year ago

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.
0:12

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,801 views • 1 year ago

Announcing 2 new members of the *St3R family for human-centric 3D vision tasks! Meet HAMst3R & HOSt3R #ICCV2025 #iccv2025 - HAMSt3R (Human-Aware Multi-view Stereo 3D Reconstruction) extends MASt3R to handle scenes involving people. Paper: 1/2 🧵
0:10

Announcing 2 new members of the *St3R family for human-centric 3D vision tasks! Meet HAMst3R & HOSt3R #ICCV2025 #iccv2025 - HAMSt3R (Human-Aware Multi-view Stereo 3D Reconstruction) extends MASt3R to handle scenes involving people. Paper: 1/2 🧵

NAVER LABS Europe

14,057 views • 9 months ago

I can no longer walk around a city without seeing a machine-readable 3D model in my head. This is a geometric and semantic 3D model of San Francisco. These maps connect the world of bits and atoms, enabling visual positioning, 3D navigation and geospatial intelligence.
0:12

I can no longer walk around a city without seeing a machine-readable 3D model in my head. This is a geometric and semantic 3D model of San Francisco. These maps connect the world of bits and atoms, enabling visual positioning, 3D navigation and geospatial intelligence.

Bilawal Sidhu

260,180 views • 1 year ago

Explore the depth of the neck with #SciePro's 3D models! The deep neck muscles are key to movement and stability. Perfect for #MedEd and healthcare pros. #science #anatomy #3d
0:16

Explore the depth of the neck with #SciePro's 3D models! The deep neck muscles are key to movement and stability. Perfect for #MedEd and healthcare pros. #science #anatomy #3d

SciePro

37,777 views • 2 years ago