Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Doesn't look like the super low-res depth maps from the iPhone Pro's LiDAR sensor would be too useful as supervision (or sanity check) for recent monocular depth estimators.

Chris Offner

4,609 subscribers

66,068 views • 1 year ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Let me share how the maps look like. Normal was estimated from the source image with StableNormal ( and Refined Depth came from promptDA ( + Lidar Depth. If you pause the video, you will see the differences in depths.

Let me share how the maps look like. Normal was estimated from the source image with StableNormal ( and Refined Depth came from promptDA ( + Lidar Depth. If you pause the video, you will see the differences in depths.

Chris make some 3D scans

44,280 views • 1 year ago

AI-Enhanced LiDAR. Left vs right. A real LiDAR device costs thousands of dollars. What's in your iPhone Pro is a "baby LiDAR". Limited depth resolution, noisy output, not really built for high-precision 3D. You can't change the hardware. So we built an ML layer on top. Denoising, geometry completion, detail recovery. Processed server-side. Same sensor. Very different result.

AI-Enhanced LiDAR. Left vs right. A real LiDAR device costs thousands of dollars. What's in your iPhone Pro is a "baby LiDAR". Limited depth resolution, noisy output, not really built for high-precision 3D. You can't change the hardware. So we built an ML layer on top. Denoising, geometry completion, detail recovery. Processed server-side. Same sensor. Very different result.

KIRI Engine - 3D Scanner App

51,682 views • 2 months ago

Some updates on the multiview vistadream pipeline with Rerun! Rerun came in extremely useful here, as being able to visualize depths at each stage of the pipeline allowed me to debug some nasty bugs. Since the last time, I was only working with a single image input. I've added in VGGT as my multiview pose + depth estimator. It works REALLY well for getting camera poses, but the depths are not that great. To try and fix that, I estimated depth maps from MoGeV2 for each of the views, and scale+shift aligned them so that they would match up to the confident sections of VGGT's depth predictions. You can see in the video just how much sharper the visualized 2d depth maps are! The biggest issue continues to be the multiview consistency 🫠 That's up next, along with actually training the Gaussian splat. Lots of work went into actually understanding inputs+outputs for VGGT. I had some funky bugs where the confidence values would all collapse to true I'm also really excited for this pipeline to use Difix3D+ Nvidia instead of Flux Inpainting, it seems like a better suited for a multiview pipeline.

Some updates on the multiview vistadream pipeline with Rerun! Rerun came in extremely useful here, as being able to visualize depths at each stage of the pipeline allowed me to debug some nasty bugs. Since the last time, I was only working with a single image input. I've added in VGGT as my multiview pose + depth estimator. It works REALLY well for getting camera poses, but the depths are not that great. To try and fix that, I estimated depth maps from MoGeV2 for each of the views, and scale+shift aligned them so that they would match up to the confident sections of VGGT's depth predictions. You can see in the video just how much sharper the visualized 2d depth maps are! The biggest issue continues to be the multiview consistency 🫠 That's up next, along with actually training the Gaussian splat. Lots of work went into actually understanding inputs+outputs for VGGT. I had some funky bugs where the confidence values would all collapse to true I'm also really excited for this pipeline to use Difix3D+ Nvidia instead of Flux Inpainting, it seems like a better suited for a multiview pipeline.

Pablo Vela

29,904 views • 11 months ago

If you use a Valve Index: The new SteamVR 2.8 Beta’s Theater Mode overlay also has a toggle to switch from VR to Passthrough while you use it But of course, you don’t get the reflection floor mapping in Passthrough. Need a depth buffer for something like that to be useful!

If you use a Valve Index: The new SteamVR 2.8 Beta’s Theater Mode overlay also has a toggle to switch from VR to Passthrough while you use it But of course, you don’t get the reflection floor mapping in Passthrough. Need a depth buffer for something like that to be useful!

SadlyItsDadley

17,325 views • 2 years ago

I was today years old when I found out Apple Maps works even offline in flight mode It's super fun and easy to look down from the plane and know what city or country you're looking at The landing was fun too I'm like a child when I'm on a plane, sorry 🙃

I was today years old when I found out Apple Maps works even offline in flight mode It's super fun and easy to look down from the plane and know what city or country you're looking at The landing was fun too I'm like a child when I'm on a plane, sorry 🙃

stemonte

2,430,845 views • 1 month ago

you don't need threejs for this effect just apply `perspective` to the container and animate each image with `translate3d(...)` over time, where depth is something like `z = z0 + speed * dt` and the rest is handled by the browser easy to map opacity or scroll to speed too

you don't need threejs for this effect just apply `perspective` to the container and animate each image with `translate3d(...)` over time, where depth is something like `z = z0 + speed * dt` and the rest is handled by the browser easy to map opacity or scroll to speed too

daniel petho

116,696 views • 5 months ago

Dear artist friends working w/ AI, I can now confirm that AI art is art :) Our piece at MoMA got reviews from 3 biased art critiques using same descriptions with super shallow and like catchy titles. At the meantime 6 true researcher critiques looked for the depth and meaning. :)

Dear artist friends working w/ AI, I can now confirm that AI art is art :) Our piece at MoMA got reviews from 3 biased art critiques using same descriptions with super shallow and like catchy titles. At the meantime 6 true researcher critiques looked for the depth and meaning. :)

Refik Anadol

1,138,738 views • 3 years ago

China just dropped an open source model that turns any phone into a 3D scanner you point it at a room and walk through it the whole space builds itself into a 3D point cloud in real time as the camera moves, no LiDAR and no depth sensor anywhere it held a full 13 minute walkthrough at 25,000 frames without the map drifting or collapsing > ~20 fps on a single GPU > works indoors and outdoors from plain footage > the scan normally needs a $50,000 rig and a technician > Apache 2.0 and runs fully offline on your machine repo in the reply

China just dropped an open source model that turns any phone into a 3D scanner you point it at a room and walk through it the whole space builds itself into a 3D point cloud in real time as the camera moves, no LiDAR and no depth sensor anywhere it held a full 13 minute walkthrough at 25,000 frames without the map drifting or collapsing > ~20 fps on a single GPU > works indoors and outdoors from plain footage > the scan normally needs a $50,000 rig and a technician > Apache 2.0 and runs fully offline on your machine repo in the reply

savip

104,123 views • 22 days ago

A preview of what's next, visualized with Rerun and PlayCanvas supersplat ✨ (Also, feel free to send me a DM 📩; I’ll be in San Francisco from July 21–29, and I'm looking to meet like-minded folks!) I'm convinced that Gaussian Splats will be an integral part of any data engine as an underlying representation. So I've started putting together a repo that: 1. Given a single image, perform image outpainting 🖼️🖌️ 2. Estimate a monocular depth map on the outpainted image 📏 3. Train a Gaussian Splat initialized from the monocular depth 🎓✨ 4. Warp to new views, perform inpainting on the missing masks -> Train new splat 🔄🎨 This is going to be integrated into exo-egoforge, but I wanted to start with the simple single-image version before moving to a multi-video implementation There's some weirdness in the final rerun visualization, but the trained splat looks great 🎉! This is all based on the very cool VistaDream paper ( .github.io/) More on this next week!

A preview of what's next, visualized with Rerun and PlayCanvas supersplat ✨ (Also, feel free to send me a DM 📩; I’ll be in San Francisco from July 21–29, and I'm looking to meet like-minded folks!) I'm convinced that Gaussian Splats will be an integral part of any data engine as an underlying representation. So I've started putting together a repo that: 1. Given a single image, perform image outpainting 🖼️🖌️ 2. Estimate a monocular depth map on the outpainted image 📏 3. Train a Gaussian Splat initialized from the monocular depth 🎓✨ 4. Warp to new views, perform inpainting on the missing masks -> Train new splat 🔄🎨 This is going to be integrated into exo-egoforge, but I wanted to start with the simple single-image version before moving to a multi-video implementation There's some weirdness in the final rerun visualization, but the trained splat looks great 🎉! This is all based on the very cool VistaDream paper ( .github.io/) More on this next week!

Pablo Vela

26,036 views • 1 year ago

名画を立体的に鑑賞する方法 Nano Banana Pro (深度/法線マップ生成)→ WebGL(Three.js) スマホでも体験できます👉 【プロンプト】（法線マップ用） Generate a surface normal map from the input artwork image. Output one RGB image where each pixel encodes the surface normal direction in standard tangent-space format: •R = X direction •G = Y direction •B = Z direction Requirements: •Maintain the same resolution and aspect ratio as the input image. •Normal directions must follow the local geometry implied by brush strokes, shading, and contours. •Do NOT add new artistic elements or repaint the scene. •Only infer plausible micro-geometry and surface orientation. •Ensure smooth, consistent normal transitions without noise. （深度マップ用） Generate a depth map from the input artwork image. Output a single grayscale depth map where: •White = near to the viewer •Black = far from the viewer Requirements: •Maintain the same resolution and aspect ratio as the input image. •Depth must be smooth, continuous, and physically plausible. •Infer only geometric depth that could reasonably exist in the original artwork. •Do NOT add new objects or details. •Do NOT stylize, repaint, or reinterpret the scene. •Avoid texture noise; focus solely on depth structure.

名画を立体的に鑑賞する方法 Nano Banana Pro (深度/法線マップ生成)→ WebGL(Three.js) スマホでも体験できます👉 【プロンプト】（法線マップ用） Generate a surface normal map from the input artwork image. Output one RGB image where each pixel encodes the surface normal direction in standard tangent-space format: •R = X direction •G = Y direction •B = Z direction Requirements: •Maintain the same resolution and aspect ratio as the input image. •Normal directions must follow the local geometry implied by brush strokes, shading, and contours. •Do NOT add new artistic elements or repaint the scene. •Only infer plausible micro-geometry and surface orientation. •Ensure smooth, consistent normal transitions without noise. （深度マップ用） Generate a depth map from the input artwork image. Output a single grayscale depth map where: •White = near to the viewer •Black = far from the viewer Requirements: •Maintain the same resolution and aspect ratio as the input image. •Depth must be smooth, continuous, and physically plausible. •Infer only geometric depth that could reasonably exist in the original artwork. •Do NOT add new objects or details. •Do NOT stylize, repaint, or reinterpret the scene. •Avoid texture noise; focus solely on depth structure.

ハシモトタクマ / FunTech

27,192 views • 8 months ago

Pacifico seems to be about the same height as Yamal, or maybe a little taller, but he definitely doesn't look like he's 1.87m 🤨 For reference, the person giving coaching instructions next to Pacifico is Heiko Westerman(1.90 m), the tallest on Barcelona's coaching staff ↕️

Pacifico seems to be about the same height as Yamal, or maybe a little taller, but he definitely doesn't look like he's 1.87m 🤨 For reference, the person giving coaching instructions next to Pacifico is Heiko Westerman(1.90 m), the tallest on Barcelona's coaching staff ↕️

David 🇰🇷

50,159 views • 4 months ago

So apparently Apple has begun production of a variable aperture camera for the iPhone 18 Pro Max. This would let the lens physically change its aperture depending on lighting, opening up in low light and narrowing in bright scenes. The result should be better exposure control and more natural depth of field. Korean publication ETNews reports that Sunny Optical has already started producing the tiny actuators responsible for this movement with full camera module assembly expected to kick off in early summer. It’s happening! Concept video by fpt./YouTube

So apparently Apple has begun production of a variable aperture camera for the iPhone 18 Pro Max. This would let the lens physically change its aperture depending on lighting, opening up in low light and narrowing in bright scenes. The result should be better exposure control and more natural depth of field. Korean publication ETNews reports that Sunny Optical has already started producing the tiny actuators responsible for this movement with full camera module assembly expected to kick off in early summer. It’s happening! Concept video by fpt./YouTube

Shishir

183,860 views • 3 months ago

For the facial animations in my game, I use a technique that yields results similar to L.A. Noire. To keep it simple: I first generate a facial animation video using AI like LTX 2. Then, I create a video depth map from that animation and project it onto a face mask. Using vertex displacement, the depth map dynamically deforms the face as the character speaks. The result looks good in standalone VR. I actually created this workflow long time ago:

For the facial animations in my game, I use a technique that yields results similar to L.A. Noire. To keep it simple: I first generate a facial animation video using AI like LTX 2. Then, I create a video depth map from that animation and project it onto a face mask. Using vertex displacement, the depth map dynamically deforms the face as the character speaks. The result looks good in standalone VR. I actually created this workflow long time ago:

Alex

48,296 views • 5 months ago

🇨🇳 Saw my first actually useful robot You know those toll way machines and you're always too far off the machine and have to hang out the window to pay, or you're too close and your wheel hits the side In Beijing they have a robot arm for it Although to be fair in Europe we just have a sensor we glue to the windshield and it auto charges your bank via IBAN So maybe not so useful But it looks cool!

🇨🇳 Saw my first actually useful robot You know those toll way machines and you're always too far off the machine and have to hang out the window to pay, or you're too close and your wheel hits the side In Beijing they have a robot arm for it Although to be fair in Europe we just have a sensor we glue to the windshield and it auto charges your bank via IBAN So maybe not so useful But it looks cool!

@levelsio

849,540 views • 8 months ago

The face doesn't look like Tifa anymore. 😅 I guess reducing the movement would help, but that wouldn’t be as fun. And 5 seconds isn’t enough! I only have one more attempt left this month. (Remaining credits: 21)🥹

The face doesn't look like Tifa anymore. 😅 I guess reducing the movement would help, but that wouldn’t be as fun. And 5 seconds isn’t enough! I only have one more attempt left this month. (Remaining credits: 21)🥹

t-fAI

17,325 views • 1 year ago

This is the only way you’ll feel when you buy from me cause I too dey deliver. check out business page @Royeshoes for shoes you’ll like or send a dm with a request as what’s posted isn’t all we have.

This is the only way you’ll feel when you buy from me cause I too dey deliver. check out business page @Royeshoes for shoes you’ll like or send a dm with a request as what’s posted isn’t all we have.

A Y O | 👟🔌

15,409 views • 3 months ago

I would love if epic added some of the movement items such as Grapple Blade, Rocket Ram, Rocket Drill or shockwave hammer to STW. they can be flex with consumable items in the 4th slot and be draftable from the inventory with a low damage output and working off durability

I would love if epic added some of the movement items such as Grapple Blade, Rocket Ram, Rocket Drill or shockwave hammer to STW. they can be flex with consumable items in the 4th slot and be draftable from the inventory with a low damage output and working off durability

JoellJoelly | Fortnite: Save The World.

36,835 views • 6 months ago

Robots can now reconstruct 3D scenes in real time from a single RGB camera. [📍 Projects page + paper] No depth sensor. No retraining. 30 FPS. Researchers at the Imperial College London introduced KV-Tracker, a training-free method that makes heavy models like π³ and Depth Anything 3 fast enough for real-time tracking. The idea is simple. These models use global self-attention, which is powerful but computationally expensive. KV-Tracker caches the key and value pairs from selected keyframes and reuses them for new frames. That cache becomes an implicit scene representation. Result: • Up to 30 FPS • 10 to 15x speedup • Accurate 6-DoF tracking on benchmarks like TUM RGB-D and 7-Scenes • Works with monocular RGB only It also supports object-level tracking with masks and allows saving the KV-cache for later reuse. For robotics, this reduces hardware constraints and moves real-time 3D perception closer to practical deployment. Credit to Marwan Taher (Marwan Taher) at Imperial’s Dyson Robotics Lab and many others who contributed to this! 📍 Save projects page + paper for later: Video: ——- if it matters in AI or Robotics you'll read it here first:

Robots can now reconstruct 3D scenes in real time from a single RGB camera. [📍 Projects page + paper] No depth sensor. No retraining. 30 FPS. Researchers at the Imperial College London introduced KV-Tracker, a training-free method that makes heavy models like π³ and Depth Anything 3 fast enough for real-time tracking. The idea is simple. These models use global self-attention, which is powerful but computationally expensive. KV-Tracker caches the key and value pairs from selected keyframes and reuses them for new frames. That cache becomes an implicit scene representation. Result: • Up to 30 FPS • 10 to 15x speedup • Accurate 6-DoF tracking on benchmarks like TUM RGB-D and 7-Scenes • Works with monocular RGB only It also supports object-level tracking with masks and allows saving the KV-cache for later reuse. For robotics, this reduces hardware constraints and moves real-time 3D perception closer to practical deployment. Credit to Marwan Taher (Marwan Taher) at Imperial’s Dyson Robotics Lab and many others who contributed to this! 📍 Save projects page + paper for later: Video: ——- if it matters in AI or Robotics you'll read it here first:

Ilir Aliu

53,911 views • 4 months ago

Just 29 clips of a normal day in tier 2 tournaments. All of them from the same guy in 2 maps ONLY. We start with an appetizer for you to check how bot he is to be that good. - Doesn't even know how to jump out of heaven, doesn't molly antirush B vs forcebuy and hides barrels.

Just 29 clips of a normal day in tier 2 tournaments. All of them from the same guy in 2 maps ONLY. We start with an appetizer for you to check how bot he is to be that good. - Doesn't even know how to jump out of heaven, doesn't molly antirush B vs forcebuy and hides barrels.

Juve

492,248 views • 11 months ago

Look at the Genius Idea He Had for These Old Barrels 🔨🌿 He turned barrels nobody wanted into something creative, useful, and full of charm. It is amazing what a little vision can do with forgotten materials. Would you try something like this too? 🛢️

Look at the Genius Idea He Had for These Old Barrels 🔨🌿 He turned barrels nobody wanted into something creative, useful, and full of charm. It is amazing what a little vision can do with forgotten materials. Would you try something like this too? 🛢️

Worldly

11,701 views • 13 days ago