Recently, I've been playing with my iPhone ToF sensor, but the problem has always been the abysmal resolution (256x192). The team behind DepthAnything released PromptDepthAnything that fixes this. Using polycam to collect the raw data, Gradio to generate a UI, and Rerun to visualize. Links at the end of... the threadshow more

There's been a lot of progress with monocular depth estimators but most often they are either relative depth maps, or if they do provide metric depth, they are not accurate enough to be useful. Prompt Depth Anything provides metric depth maps at up to 4k resolution and much higher detail/fidelity and still very accurate

Often when looking at depth maps, it's easy to be fooled by how they look in 2D, so I wanted to use @Polycam3D to make sure I could un-project them into 3D and see if the reconstruction seems reasonable.

On all my testing thus far, Prompt Depth Anything seems to generalize super well, the only failure points I noticed have to do with difficult reflective surfaces such as windows or mirrors. It also seems that it's only really built for iPhone lidars, but I'm sure it would be fairly easy to fine-tune to a different use case like they showed with the robotics example

My goal from here is to figure out if I can reproduce the robotics grasping example using an iPhone lidar with so100-arm from @LeRobotHF and potentially a quest3 for teleoperation. This is a much larger project so I'll break it down into manageable chunks and I will share the code for those who want to follow along

Links to the repo and easy to view example

I also wanted to make sure I thanked the Prompt Depth Anything team for open-sourcing this great resource @HaotongLin @songyoupeng @JiamingSuen @ericliuof97 @XiaoweiZhou5 @bingyikang

Guess who’s not a fan of data ownership? 😅 When #Zuckerberg gives zero stars, you know the @datamaskwallet app is keeping your data safe from the big tech giants' grasp. 🚫📊 #DataPrivacy #BigTechNemesis art and creative by @Matteo_Frog

@Polycam3D @Gradio Wait do I have one of these on my 16?

@Polycam3D @Gradio Only pro models sadly, so if you have a 16pro yes! but not the regular 16

@Polycam3D @Gradio for the lulz.

@Polycam3D @Gradio Have you tried this with the DepthPro model from Apple? It also produces a metric depth map in high resolution without knowledge of the camera intrinsics. I was able to generate very accurate meshes with it

@Polycam3D @Gradio I have not tried it! Though I now there was some ablation studies in the paper comparing them. They show an example with Metric3Dv2