Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

InfiniDepth: Fine-grained depth estimation at any resolution; models depth as a neural implicit field instead of a grid; - tops DepthAnything; - recovers sharp details w/ a lightweight 15M decoder.

wildmindai's profile picture

Wildminder

12,378 subscribers

59,393 Aufrufe • vor 9 Monaten •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

4 Kommentare

Profilbild von WhatAboutFrank
WhatAboutFrankvor 9 Monaten

This is awesome. The problem with all these has always been: what is ‘behind’ the object who’s depth your estimating. It ends up being just ‘empty space’. Could we combine this with some networks that train on different 3d environments that learn how to fill these gaps the same way use generative ai?

Profilbild von Matthias Schmidt
Matthias Schmidtvor 9 Monaten

Why does nobody create a model that outputs better bit depth? That would be so much more useful. 16k at 8 bit you still have banding and other artifacts everywhere.

Profilbild von utsab
utsabvor 9 Monaten

15M decoder?! seriously?

Profilbild von WhatAboutFrank
WhatAboutFrankvor 9 Monaten

Wild idea, apply this to the Minnesota driver

Ähnliche Videos

Excited to share our work InfiniDepth (CVPR 2026) — casting monocular depth estimation as a neural implicit field, which enables: 🔍 Arbitrary-Resolution 📐 Accurate Metric Depth 📷 Large-View Novel View Synthesis Feel free to try our code:
0:34

Excited to share our work InfiniDepth (CVPR 2026) — casting monocular depth estimation as a neural implicit field, which enables: 🔍 Arbitrary-Resolution 📐 Accurate Metric Depth 📷 Large-View Novel View Synthesis Feel free to try our code:

Sida Peng

39,053 Aufrufe • vor 7 Monaten

📢Panoptic Lifting for 3D Scene Understanding with Neural Fields #CVPR23 highlight! Given only posed RGB images of a scene, we optimize a panoptic radiance field representing color, depth, semantics, and instances at any point in space. Vid: Yawar Siddiqui
0:44

📢Panoptic Lifting for 3D Scene Understanding with Neural Fields #CVPR23 highlight! Given only posed RGB images of a scene, we optimize a panoptic radiance field representing color, depth, semantics, and instances at any point in space. Vid: Yawar Siddiqui

Matthias Niessner

29,937 Aufrufe • vor 3 Jahren

After a year of team work, we're thrilled to introduce Depth Anything 3 (DA3)! 🚀 Aiming for human-like spatial perception, DA3 extends monocular depth estimation to any-view scenarios, including single images, multi-view images, and video. In pursuit of minimal modeling, DA3 reveals two key insights: 💎 A plain transformer (e.g., vanilla DINO) is enough. No specialized architecture. ✨ A single depth-ray representation is enough. No complex 3D tasks. Three series of models have been released: the main DA3 series, a monocular metric estimation series, and a monocular depth estimation series. The core team members, aside from me: Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen. 👇(1/n) #DepthAnything3
0:32

After a year of team work, we're thrilled to introduce Depth Anything 3 (DA3)! 🚀 Aiming for human-like spatial perception, DA3 extends monocular depth estimation to any-view scenarios, including single images, multi-view images, and video. In pursuit of minimal modeling, DA3 reveals two key insights: 💎 A plain transformer (e.g., vanilla DINO) is enough. No specialized architecture. ✨ A single depth-ray representation is enough. No complex 3D tasks. Three series of models have been released: the main DA3 series, a monocular metric estimation series, and a monocular depth estimation series. The core team members, aside from me: Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen. 👇(1/n) #DepthAnything3

Bingyi Kang

516,444 Aufrufe • vor 11 Monaten

Presenting MonoNeRF at #ICML2023 We train a generalizable NeRF from: ✅Large-scale monocular videos instead of one scene ✅No GT camera poses.📷🚫 Without per-scene optimization, the model can do view synthesis, depth estimation, camera pose estimation.
1:56

Presenting MonoNeRF at #ICML2023 We train a generalizable NeRF from: ✅Large-scale monocular videos instead of one scene ✅No GT camera poses.📷🚫 Without per-scene optimization, the model can do view synthesis, depth estimation, camera pose estimation.

Xiaolong Wang

36,588 Aufrufe • vor 3 Jahren

Want to use Depth Anything, but need metric depth rather than relative depth? Thrilled to introduce Prompt Depth Anything, a new paradigm for accurate metric depth estimation with up to 4K resolution. 👉Key Message: Depth foundation models like DA have already internalized rich geometric knowledge of the 3D world but lack a proper way to elicit it. Inspired by the success of prompting in LLMs, we propose prompting Depth Anything with metric cues to produce metric depth. This method proves to be very effective when using a low-cost lidar (e.g., iPhone's LiDAR), which is widely available, as prompts. We believe the prompt can generalize to other forms as long as scale information is provided. Prompt Depth Anything offers 1⃣A series of models for iPhone lidars. 2⃣4D reconstruction from monocular videos (captured with iPhone). 3⃣Improved generalization ability for robot manipulation, e.g. Training on cans but generalizing on glasses. 4⃣More detailed depth annotations for the ScanNet++ dataset. The first author is our excellent intern Haotong Lin. Paper: Huggingface: Project Page: Code:
1:20

Want to use Depth Anything, but need metric depth rather than relative depth? Thrilled to introduce Prompt Depth Anything, a new paradigm for accurate metric depth estimation with up to 4K resolution. 👉Key Message: Depth foundation models like DA have already internalized rich geometric knowledge of the 3D world but lack a proper way to elicit it. Inspired by the success of prompting in LLMs, we propose prompting Depth Anything with metric cues to produce metric depth. This method proves to be very effective when using a low-cost lidar (e.g., iPhone's LiDAR), which is widely available, as prompts. We believe the prompt can generalize to other forms as long as scale information is provided. Prompt Depth Anything offers 1⃣A series of models for iPhone lidars. 2⃣4D reconstruction from monocular videos (captured with iPhone). 3⃣Improved generalization ability for robot manipulation, e.g. Training on cans but generalizing on glasses. 4⃣More detailed depth annotations for the ScanNet++ dataset. The first author is our excellent intern Haotong Lin. Paper: Huggingface: Project Page: Code:

Bingyi Kang

67,750 Aufrufe • vor 1 Jahr

Implicit Neural Light Spheres lets you turn panoramic captures into dynamic wide FOV renders (with real-time rendering!). Instead of generating panoramas with image stitching, we use neural light spheres to jointly estimate the camera path and a high-resolution scene reconstruction to produce novel wide field-of-view projections of the environment. Code, data, and info: Amazing work with Ilya Chugunov, Amogh Joshi, Kiran Murthy, François Bleibel
0:51

Implicit Neural Light Spheres lets you turn panoramic captures into dynamic wide FOV renders (with real-time rendering!). Instead of generating panoramas with image stitching, we use neural light spheres to jointly estimate the camera path and a high-resolution scene reconstruction to produce novel wide field-of-view projections of the environment. Code, data, and info: Amazing work with Ilya Chugunov, Amogh Joshi, Kiran Murthy, François Bleibel

Felix Heide

10,312 Aufrufe • vor 2 Jahren

Check out the Neural Graph Mapping for Dense SLAM with Efficient Loop Closure paper by Leonard Bruns, Jun Zhang, and Patric Jensfelt, visualized with Rerun. “Existing neural field-based SLAM methods typically employ a single monolithic field as their scene representation. This prevents efficient incorporation of loop closure constraints and limits scalability. To address these shortcomings, we propose a neural mapping framework which anchors lightweight neural fields to the pose graph of a sparse visual SLAM system.”
0:37

Check out the Neural Graph Mapping for Dense SLAM with Efficient Loop Closure paper by Leonard Bruns, Jun Zhang, and Patric Jensfelt, visualized with Rerun. “Existing neural field-based SLAM methods typically employ a single monolithic field as their scene representation. This prevents efficient incorporation of loop closure constraints and limits scalability. To address these shortcomings, we propose a neural mapping framework which anchors lightweight neural fields to the pose graph of a sparse visual SLAM system.”

Rerun

12,865 Aufrufe • vor 2 Jahren

Made this video (🎶) with a Midjourney v6 image! Started by upscaling/refining with Magnific.ai, pulled a Marigold Depth Map from that in ComfyUI, then used as a displacement map in Blender where I animated this camera pass with some relighting and narrow depth of field.🧵1/12
0:55

Made this video (🎶) with a Midjourney v6 image! Started by upscaling/refining with Magnific.ai, pulled a Marigold Depth Map from that in ComfyUI, then used as a displacement map in Blender where I animated this camera pass with some relighting and narrow depth of field.🧵1/12

CoffeeVectors

135,713 Aufrufe • vor 2 Jahren

If your photo has lost quality, it's not the end of the world. Give it a second chance: details, sharpness, depth. Experiment yourself at 💡
0:30

If your photo has lost quality, it's not the end of the world. Give it a second chance: details, sharpness, depth. Experiment yourself at 💡

nurse_natalie

20,034 Aufrufe • vor 9 Monaten

Meet MapAnything – a transformer that directly regresses factored metric 3D scene geometry (from images, calibration, poses, or depth) in an end-to-end way. No pipelines, no extra stages. Just 3D geometry & cameras, straight from any type of input, delivering new state-of-the-art results 🚀 One universal model enables SoTA for: 🔥 Mono Depth Estimation 🔥 Multi-View SfM 🔥 Multi-View Stereo 🔥 Depth Completion 🔥 Registration … and many more possibilities! – plus everything is metric 🎯 We release code for data processing, training, benchmarking & ablations – everything Apache 2.0! Details & Links 👇
1:30

Meet MapAnything – a transformer that directly regresses factored metric 3D scene geometry (from images, calibration, poses, or depth) in an end-to-end way. No pipelines, no extra stages. Just 3D geometry & cameras, straight from any type of input, delivering new state-of-the-art results 🚀 One universal model enables SoTA for: 🔥 Mono Depth Estimation 🔥 Multi-View SfM 🔥 Multi-View Stereo 🔥 Depth Completion 🔥 Registration … and many more possibilities! – plus everything is metric 🎯 We release code for data processing, training, benchmarking & ablations – everything Apache 2.0! Details & Links 👇

Nikhil Keetha

123,693 Aufrufe • vor 1 Jahr

I've been working on a new Figma plugin called Selection Text. It's a lightweight way to see how your text layers are styled in any given frame or set of frames. I designed it to be fast, lightweight, and feel native to Figma. I'll post an in-depth look once it's published!
0:22

I've been working on a new Figma plugin called Selection Text. It's a lightweight way to see how your text layers are styled in any given frame or set of frames. I designed it to be fast, lightweight, and feel native to Figma. I'll post an in-depth look once it's published!

Daniel Destefanis

47,159 Aufrufe • vor 2 Jahren

In collaboration with Intel, our Depth Fusion showcases the power of our LDM3D diffusion model in generating 360° views from text prompts provided by the user. The LDM3D diffusion model generates a 2D RGB image and its corresponding relative depth map providing a complete RGBD representation corresponding to the text prompt. The LDM 3D model is a specialized version of the stable diffusion V 1.4 model that has been modified to fit both image and depth map data.The model was then fine tuned on a subset of the Laion400M data set - large scale image caption data set. The depth maps used to fine tune our model were generated by the DPTBeiT large 512 depth estimation model that provides highly accurate relative depth estimates for each pixel. We take the generated 2D RGB image and depth map and use them to compute a 360° projection using touchdesigner. Touchdesigner is a versatile platform that allows for the creation of immersive and interactive multimedia experiences. Our application harnesses the power of touchdesigner to bring the generated 360° views to life, providing users with a unique and engaging way to experience their text prompts, whether it’s a description of a tranquil forest, a noisy cityscape or a futuristic sci fi world. Our depth fusion can bring these concepts to life in a vivid and immersive detail. - Scottie Fox, VP Engineering Blockade Labs ScottieFox #AI #VR #3D #gamedev #stablediffusion
2:29

In collaboration with Intel, our Depth Fusion showcases the power of our LDM3D diffusion model in generating 360° views from text prompts provided by the user. The LDM3D diffusion model generates a 2D RGB image and its corresponding relative depth map providing a complete RGBD representation corresponding to the text prompt. The LDM 3D model is a specialized version of the stable diffusion V 1.4 model that has been modified to fit both image and depth map data.The model was then fine tuned on a subset of the Laion400M data set - large scale image caption data set. The depth maps used to fine tune our model were generated by the DPTBeiT large 512 depth estimation model that provides highly accurate relative depth estimates for each pixel. We take the generated 2D RGB image and depth map and use them to compute a 360° projection using touchdesigner. Touchdesigner is a versatile platform that allows for the creation of immersive and interactive multimedia experiences. Our application harnesses the power of touchdesigner to bring the generated 360° views to life, providing users with a unique and engaging way to experience their text prompts, whether it’s a description of a tranquil forest, a noisy cityscape or a futuristic sci fi world. Our depth fusion can bring these concepts to life in a vivid and immersive detail. - Scottie Fox, VP Engineering Blockade Labs ScottieFox #AI #VR #3D #gamedev #stablediffusion

Blockade Labs

11,458 Aufrufe • vor 3 Jahren

Working on room scan streaming for the Lasertag game: headsets build a shared heightmap of the playspace collaboratively in the background. Quest's depth estimation is only good up to ~5 meters, so this lets people fight at greater distances
0:43

Working on room scan streaming for the Lasertag game: headsets build a shared heightmap of the playspace collaboratively in the background. Quest's depth estimation is only good up to ~5 meters, so this lets people fight at greater distances

jt

14,474 Aufrufe • vor 1 Jahr

all of this is code Sonnet 5.5 max made this 39s video in 17 hours w/ HyperFrames , prompt below concept: depth is the hierarchy. On a screen you look into, focus picks what matters and going deeper shows more. • One object: a black glass instrument with three windows, all showing the same tide reading, 3.67 m. • Three depths, three levels: the shallowest window shows only the number, the middle one adds range, high water and rate, the deepest adds a trend and twelve readings. • The camera pulls focus from window to window, so focus, not layout, decides what you read. It ends on the engraved line "Depth is the hierarchy". • The design idea: information hierarchy carried by optical depth (glass, refraction, real depth of field) instead of size, weight and position on a flat screen. The starting point was Ferrari Luce's layered display. • Built as one HyperFrames page: three.js glass and metal, live HTML text drawn into the 3D scene through HTML-in-Canvas, thin-lens depth of field, every frame deterministic.
0:39

all of this is code Sonnet 5.5 max made this 39s video in 17 hours w/ HyperFrames , prompt below concept: depth is the hierarchy. On a screen you look into, focus picks what matters and going deeper shows more. • One object: a black glass instrument with three windows, all showing the same tide reading, 3.67 m. • Three depths, three levels: the shallowest window shows only the number, the middle one adds range, high water and rate, the deepest adds a trend and twelve readings. • The camera pulls focus from window to window, so focus, not layout, decides what you read. It ends on the engraved line "Depth is the hierarchy". • The design idea: information hierarchy carried by optical depth (glass, refraction, real depth of field) instead of size, weight and position on a flat screen. The starting point was Ferrari Luce's layered display. • Built as one HyperFrames page: three.js glass and metal, live HTML text drawn into the 3D scene through HTML-in-Canvas, thin-lens depth of field, every frame deterministic.

Miguel Ángel

30,121 Aufrufe • vor 22 Tagen

Meta presents Sapiens Foundation for Human Vision Models discuss: We present Sapiens, a family of models for four fundamental human-centric vision tasks - 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction. Our models natively support 1K high-resolution inference and are extremely easy to adapt for individual tasks by simply fine-tuning models pretrained on over 300 million in-the-wild human images. We observe that, given the same computational budget, self-supervised pretraining on a curated dataset of human images significantly boosts the performance for a diverse set of human-centric tasks. The resulting models exhibit remarkable generalization to in-the-wild data, even when labeled data is scarce or entirely synthetic. Our simple model design also brings scalability - model performance across tasks improves as we scale the number of parameters from 0.3 to 2 billion. Sapiens consistently surpasses existing baselines across various human-centric benchmarks. We achieve significant improvements over the prior state-of-the-art on Humans-5K (pose) by 7.6 mAP, Humans-2K (part-seg) by 17.1 mIoU, Hi4D (depth) by 22.4% relative RMSE, and THuman2 (normal) by 53.5% relative angular error.
2:00

Meta presents Sapiens Foundation for Human Vision Models discuss: We present Sapiens, a family of models for four fundamental human-centric vision tasks - 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction. Our models natively support 1K high-resolution inference and are extremely easy to adapt for individual tasks by simply fine-tuning models pretrained on over 300 million in-the-wild human images. We observe that, given the same computational budget, self-supervised pretraining on a curated dataset of human images significantly boosts the performance for a diverse set of human-centric tasks. The resulting models exhibit remarkable generalization to in-the-wild data, even when labeled data is scarce or entirely synthetic. Our simple model design also brings scalability - model performance across tasks improves as we scale the number of parameters from 0.3 to 2 billion. Sapiens consistently surpasses existing baselines across various human-centric benchmarks. We achieve significant improvements over the prior state-of-the-art on Humans-5K (pose) by 7.6 mAP, Humans-2K (part-seg) by 17.1 mIoU, Hi4D (depth) by 22.4% relative RMSE, and THuman2 (normal) by 53.5% relative angular error.

AK

151,716 Aufrufe • vor 2 Jahren

AAA cinematic depth of field with proper bokeh in three.js has been a white whale of mine for a while. I've thrown every new model release at it over the years. Astra has gotten closer than any model before. Still a few artifacts left but the model is working through them methodically. Three.js WebGL / WebGPU OpenAI
0:20

AAA cinematic depth of field with proper bokeh in three.js has been a white whale of mine for a while. I've thrown every new model release at it over the years. Astra has gotten closer than any model before. Still a few artifacts left but the model is working through them methodically. Three.js WebGL / WebGPU OpenAI

Aaron Lemke

11,089 Aufrufe • vor 1 Monat

#Gators commit Rylan McMahan (‘27, FL) struck out seven over four no-hit innings of work. Got a lively FB w/ late ASR up to 95 (92-94). Went to a 🔨 1-7 CB w/ sharp tilt + late depth. Easy operation from the left side, utilizing a pronounced LL + quick AA. #WWBA Perfect Game Florida
0:37

#Gators commit Rylan McMahan (‘27, FL) struck out seven over four no-hit innings of work. Got a lively FB w/ late ASR up to 95 (92-94). Went to a 🔨 1-7 CB w/ sharp tilt + late depth. Easy operation from the left side, utilizing a pronounced LL + quick AA. #WWBA Perfect Game Florida

Perfect Game Scout

40,415 Aufrufe • vor 3 Monaten

A. J. Hinch has a vast array of tools at his disposal! 🔧 Mark DeRosa dives in on the Detroit Tigers depth as they hold a 14-game lead in the AL Central.
3:07

A. J. Hinch has a vast array of tools at his disposal! 🔧 Mark DeRosa dives in on the Detroit Tigers depth as they hold a 14-game lead in the AL Central.

MLB Network

66,213 Aufrufe • vor 1 Jahr