Doesn't look like the super low-res depth maps from... the iPhone Pro's LiDAR sensor would be too useful as supervision (or sanity check) for recent monocular depth estimators.show more

Let me share how the maps look like. Normal... was estimated from the source image with StableNormal ( and Refined Depth came from promptDA ( + Lidar Depth. If you pause the video, you will see the differences in depths.show more

For a more in-depth look at #MW3 Multiplayer Maps 👉

VFX using iPhone LiDAR. The great thing about DoF... particles is that they easily add a sense of depth to the effects.show more

Introducing 🛹 RollingDepth 🛹 — a universal monocular depth... estimator for arbitrarily long videos! Our paper, “Video Depth without Video Models,” delivers exactly that, setting new standards in temporal consistency. Check out more details in the thread 🧵show more

AI-Enhanced LiDAR. Left vs right. A real LiDAR device... costs thousands of dollars. What's in your iPhone Pro is a "baby LiDAR". Limited depth resolution, noisy output, not really built for high-precision 3D. You can't change the hardware. So we built an ML layer on top. Denoising, geometry completion, detail recovery. Processed server-side. Same sensor. Very different result.show more

Google just revealed an ABSOLUTE depth estimation model 🤯... As opposed to recent depth models (Marigold, PatchFusion) which aim for maximum details, DMD aims to estimate the ABSOLUTE depth (in meters) within the image More details below ⬇️⬇️show more

AnyDepth: Lightweight zero-shot monocular depth estimation; surpasses DPT; -... nicely preserves detail.show more

Always check the depth before jumping

Some updates on the multiview vistadream pipeline with Rerun!... Rerun came in extremely useful here, as being able to visualize depths at each stage of the pipeline allowed me to debug some nasty bugs. Since the last time, I was only working with a single image input. I've added in VGGT as my multiview pose + depth estimator. It works REALLY well for getting camera poses, but the depths are not that great. To try and fix that, I estimated depth maps from MoGeV2 for each of the views, and scale+shift aligned them so that they would match up to the confident sections of VGGT's depth predictions. You can see in the video just how much sharper the visualized 2d depth maps are! The biggest issue continues to be the multiview consistency 🫠 That's up next, along with actually training the Gaussian splat. Lots of work went into actually understanding inputs+outputs for VGGT. I had some funky bugs where the confidence values would all collapse to true I'm also really excited for this pipeline to use Difix3D+ Nvidia instead of Flux Inpainting, it seems like a better suited for a multiview pipeline.show more

Left: raw depth from the OAK-D Pro W camera... Right: depth from FoundationStereoshow more

Trying out realtime video-to-depth on an iPhone 15 using... Depth Anything V2.show more

I added a node to my custom node that... converts the depth image to a normal map! This is useful for 2D-style images where Normals Estimation doesn't work well.show more

How else would my fitness coach check my squat depth?

Video diffusion models are just overqualified depth estimators! Deterministic... single-pass depth estimation based on WanV2.1. - SOTA 5.5 AbsRel on ScanNet - data-efficient than baselines; - no temporal flicker + infinite-length estimation w/ zero scale drift.show more

Raised from the depth WIP

Revisiting this game a few months later: (i) It's... pretty impressive that general-purpose image generators can estimate meaningful depth from a single photo. (ii) Monocular depth remains pretty awful for 3D reconstruction! Here Gemini > GPT > Qwen. (At least I hope so!…🫣)show more

Spent some time this week playing with depth estimation... models. Monocular depth estimation has come a long way, but in terms of metric accuracy and geometric consistency, stereo depth still has it beat by a wide margin. DepthPro (mono) vs FoundationStereo (stereo)show more

Gave a robot 3D vision with just a regular... camera👁️ Full Tutorial: Deployed #Depth Anything V3​ on NVIDIA Robotics Jetson AGX Orin. It estimates depth from 2D images in real-time—no special sensors needed. Just need #Monocular depth estimation + #TensorRT​ optimization + #ROS2​ integration. 👉 Learn more about reComputer Robotics J5011: #TheAIHardwarePartnershow more

Another example of inverted 2 taking away one of... Johnson’s adjusted Cross concepts. Middle pole runner can take the Cross from depth. RB runs the wrong route here too. Check flat.show more