Loading video...

Video Failed to Load

Go Home

Video diffusion models are just overqualified depth estimators! Deterministic single-pass depth estimation based on WanV2.1. - SOTA 5.5 AbsRel on ScanNet - data-efficient than baselines; - no temporal flicker + infinite-length estimation w/ zero scale drift.

wildmindai's profile picture

Wildminder

10,197 subscribers

49,229 views • 2 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

🤔Applying Depth Estimation models directly to videos can result in inconsistency between frames. 💪Well, not anymore. 🔥ChronoDepth is a new approach to video depth estimation that focuses on achieving both accuracy within each frame and consistency across frames. 📜More👇
0:15

🤔Applying Depth Estimation models directly to videos can result in inconsistency between frames. 💪Well, not anymore. 🔥ChronoDepth is a new approach to video depth estimation that focuses on achieving both accuracy within each frame and consistency across frames. 📜More👇

Gradio

19,986 views • 2 years ago

Spent some time this week playing with depth estimation models. Monocular depth estimation has come a long way, but in terms of metric accuracy and geometric consistency, stereo depth still has it beat by a wide margin. DepthPro (mono) vs FoundationStereo (stereo)
0:14

Spent some time this week playing with depth estimation models. Monocular depth estimation has come a long way, but in terms of metric accuracy and geometric consistency, stereo depth still has it beat by a wide margin. DepthPro (mono) vs FoundationStereo (stereo)

FrostyFridge

27,774 views • 1 year ago

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.
0:13

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

MrNeRF

27,428 views • 1 year ago

One more turn with this #AR: added underwater transition. All real-time, based on #ML scene depth estimation. Built in Effect House.
0:15

One more turn with this #AR: added underwater transition. All real-time, based on #ML scene depth estimation. Built in Effect House.

Denis Rossiev ᯅ/acc

23,561 views • 2 years ago

Google just revealed an ABSOLUTE depth estimation model 🤯 As opposed to recent depth models (Marigold, PatchFusion) which aim for maximum details, DMD aims to estimate the ABSOLUTE depth (in meters) within the image More details below ⬇️⬇️
0:16

Google just revealed an ABSOLUTE depth estimation model 🤯 As opposed to recent depth models (Marigold, PatchFusion) which aim for maximum details, DMD aims to estimate the ABSOLUTE depth (in meters) within the image More details below ⬇️⬇️

Alex Carlier

199,515 views • 2 years ago

Discover the right 3D Geometric Foundation Model for your task—whether it’s stereo matching, multi-view depth estimation, video depth, pose estimation, semantic understanding, or novel view synthesis. Explore more insights in our #E3DBench #FoundationModel #3D #GaussianSplatting. Project Webpage:
0:14

Discover the right 3D Geometric Foundation Model for your task—whether it’s stereo matching, multi-view depth estimation, video depth, pose estimation, semantic understanding, or novel view synthesis. Explore more insights in our #E3DBench #FoundationModel #3D #GaussianSplatting. Project Webpage:

Zhiwen(Aaron) Fan

29,920 views • 1 year ago

Gave a robot 3D vision with just a regular camera👁️ Full Tutorial: Deployed #Depth Anything V3​ on NVIDIA Robotics Jetson AGX Orin. It estimates depth from 2D images in real-time—no special sensors needed. Just need #Monocular depth estimation + #TensorRT​ optimization + #ROS2​ integration. 👉 Learn more about reComputer Robotics J5011: #TheAIHardwarePartner
0:18

Gave a robot 3D vision with just a regular camera👁️ Full Tutorial: Deployed #Depth Anything V3​ on NVIDIA Robotics Jetson AGX Orin. It estimates depth from 2D images in real-time—no special sensors needed. Just need #Monocular depth estimation + #TensorRT​ optimization + #ROS2​ integration. 👉 Learn more about reComputer Robotics J5011: #TheAIHardwarePartner

Seeed Studio

16,966 views • 3 months ago

DreamActor M2.0 just dropped on Replicate Animate any character, humans, cartoons, animals, even non-humans, from a single image + driving video. No pose estimation needed. Killer results on multiples too!
0:11

DreamActor M2.0 just dropped on Replicate Animate any character, humans, cartoons, animals, even non-humans, from a single image + driving video. No pose estimation needed. Killer results on multiples too!

Replicate

17,499 views • 3 months ago

Alibaba doesn't stop cooking. They just dropped a REALTIME, infinite length video generator. Based on Wan, 20 fps, with dialogue
0:10

Alibaba doesn't stop cooking. They just dropped a REALTIME, infinite length video generator. Based on Wan, 20 fps, with dialogue

⚡AI Search⚡

77,686 views • 6 months ago

🤯 𝐒𝐥𝐢𝐜𝐞𝐝𝐢𝐭 : A revolutionary approach to zero-shot video editing✏️🎥 using text-to-image diffusion models 🎞️Slicedit achieves great temporal consistency by leveraging spatio-temporal "slices"🔀
0:10

🤯 𝐒𝐥𝐢𝐜𝐞𝐝𝐢𝐭 : A revolutionary approach to zero-shot video editing✏️🎥 using text-to-image diffusion models 🎞️Slicedit achieves great temporal consistency by leveraging spatio-temporal "slices"🔀

Gradio

19,581 views • 2 years ago

I added a node to my custom node that converts the depth image to a normal map! This is useful for 2D-style images where Normals Estimation doesn't work well.
0:16

I added a node to my custom node that converts the depth image to a normal map! This is useful for 2D-style images where Normals Estimation doesn't work well.

toyxyz

117,354 views • 1 year ago

Trying out realtime video-to-depth on an iPhone 15 using Depth Anything V2.
0:18

Trying out realtime video-to-depth on an iPhone 15 using Depth Anything V2.

Tim Field

21,835 views • 2 years ago

Doesn't look like the super low-res depth maps from the iPhone Pro's LiDAR sensor would be too useful as supervision (or sanity check) for recent monocular depth estimators.
0:13

Doesn't look like the super low-res depth maps from the iPhone Pro's LiDAR sensor would be too useful as supervision (or sanity check) for recent monocular depth estimators.

Chris Offner

66,034 views • 1 year ago

Nano is a depth-aware atmospheric haze plugin that uses ML depth estimation to add physically accurate fog and light scattering to your footage. Works *best* on log footage with visible light sources - it analyzes scene highlights then creates airlight (atmospheric scatter) and halation (light bloom) that responds to actual depth in the scene. Pretty clever approach to getting that cinematic haze look without having to pump a fog machine on set. Makes the OG Trapcode Shine look extremely dated (basically 2D light streaks masked by luminance values), and is yet way more controllable than the current crop of generative AI video-to-video tools.
0:18

Nano is a depth-aware atmospheric haze plugin that uses ML depth estimation to add physically accurate fog and light scattering to your footage. Works *best* on log footage with visible light sources - it analyzes scene highlights then creates airlight (atmospheric scatter) and halation (light bloom) that responds to actual depth in the scene. Pretty clever approach to getting that cinematic haze look without having to pump a fog machine on set. Makes the OG Trapcode Shine look extremely dated (basically 2D light streaks masked by luminance values), and is yet way more controllable than the current crop of generative AI video-to-video tools.

Bilawal Sidhu

275,142 views • 11 months ago

Uhhh, no. I could spend an hour waxing at length about alliance structures and stockpile depth, but this video just about sums that up…
0:13

Uhhh, no. I could spend an hour waxing at length about alliance structures and stockpile depth, but this video just about sums that up…

Analytica Camillus

481,920 views • 16 days ago

Based on what I played, it’s clear this will be a massively epic game in length, scope and depth. The visual fidelity on a base PS5 is ASTOUNDING, and, as you'd expect, the cinematics and story sequences are next level.
0:14

Based on what I played, it’s clear this will be a massively epic game in length, scope and depth. The visual fidelity on a base PS5 is ASTOUNDING, and, as you'd expect, the cinematics and story sequences are next level.

Geoff Keighley

257,364 views • 1 year ago

supervision-0.26.0 is out we finally released support for ViTPose and ViTPose++ pose estimation models from Hugging Face transformers link:
0:15

supervision-0.26.0 is out we finally released support for ViTPose and ViTPose++ pose estimation models from Hugging Face transformers link:

SkalskiP

112,872 views • 10 months ago

An image + a detailed normal map generated by GeoWizard --> a relightable image. Geowizard is a generative geometry estimation model that could jointly produce high-quality depth and normal with intricate details. Try our huggingface demo here: #huggingface #gradio #geowizard
0:13

An image + a detailed normal map generated by GeoWizard --> a relightable image. Geowizard is a generative geometry estimation model that could jointly produce high-quality depth and normal with intricate details. Try our huggingface demo here: #huggingface #gradio #geowizard

Xiao-Xiao Long

30,338 views • 2 years ago

Big Marigold update! Last year, we showed how to turn Stable Diffusion 2 into a SOTA depth estimator with a few synthetic samples and 2–3 days on just 1 GPU. Today's release features: 🏎️ 1-step inference 🔢 New modalities 🫣 High resolution 🧨 Diffusers support 🕹️ New demos 🧶👇
0:16

Big Marigold update! Last year, we showed how to turn Stable Diffusion 2 into a SOTA depth estimator with a few synthetic samples and 2–3 days on just 1 GPU. Today's release features: 🏎️ 1-step inference 🔢 New modalities 🫣 High resolution 🧨 Diffusers support 🕹️ New demos 🧶👇

Anton Obukhov

24,860 views • 1 year ago

ViTPose -- best open-source pose estimation model just landed to Hugging Face transformers 🕺🏻💃🏻 See how to use on the next one ⤵️
0:10

ViTPose -- best open-source pose estimation model just landed to Hugging Face transformers 🕺🏻💃🏻 See how to use on the next one ⤵️

merve

83,367 views • 1 year ago