Loading video...

Video Failed to Load

Go Home

Want to use Depth Anything, but need metric depth rather than relative depth? Thrilled to introduce Prompt Depth Anything, a new paradigm for accurate metric depth estimation with up to 4K resolution. 👉Key Message: Depth foundation models like DA have already internalized rich geometric knowledge of the 3D world...

bingyikang's profile picture

Bingyi Kang

3,524 subscribers

67,604 views • 1 year ago •via X (Twitter)

Health & WellnessScience & TechnologyEducation
LIVE

Anya Rossi• Live Now

Private livecam show

9 Comments

Bingyi Kang's profile picture
Bingyi Kang1 year ago

This project would not have been possible without the wonderful collaboration of @pengsida, Jingxiao Chen, @songyoupeng, @JiamingSuen, @ericliuof97, Hujun Bao, Jiashi Feng and @XiaoweiZhou5.

Bingyi Kang's profile picture
Bingyi Kang1 year ago

Check out for some interactive demos there:

Bingyi Kang's profile picture
Bingyi Kang1 year ago

We thank @huggingface for the generous support in serving our demo. @_akhaliq @pcuenq

Patryk Zoltowski's profile picture
Patryk Zoltowski1 year ago

Any reason why cannot use depth from front Truedepth camera? They are also available in ARKit (with head tracking config) and from my experience higher quality and bigger resolution (640x480) than those from Lidar. However only provided at 30fps.

Basid's profile picture
Basid1 year ago

exactly what I needed, good work

Redmond Hosting's profile picture
Redmond Hosting1 year ago

What about using this to upscale depth from a multi-view depth model like Depth Crafter?

Bingyi Kang's profile picture
Bingyi Kang1 year ago

The same idea can be applied to this setting. But the models released are only for iPhone's Lidar.

Xiaoyu Xiang's profile picture
Xiaoyu Xiang1 year ago

Wow, a strong baseline to compare in our upcoming paper!

hantmango's profile picture
hantmango1 year ago

Don't have Iphone....

Related Videos

After a year of team work, we're thrilled to introduce Depth Anything 3 (DA3)! 🚀 Aiming for human-like spatial perception, DA3 extends monocular depth estimation to any-view scenarios, including single images, multi-view images, and video. In pursuit of minimal modeling, DA3 reveals two key insights: 💎 A plain transformer (e.g., vanilla DINO) is enough. No specialized architecture. ✨ A single depth-ray representation is enough. No complex 3D tasks. Three series of models have been released: the main DA3 series, a monocular metric estimation series, and a monocular depth estimation series. The core team members, aside from me: Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen. 👇(1/n) #DepthAnything3
0:32

After a year of team work, we're thrilled to introduce Depth Anything 3 (DA3)! 🚀 Aiming for human-like spatial perception, DA3 extends monocular depth estimation to any-view scenarios, including single images, multi-view images, and video. In pursuit of minimal modeling, DA3 reveals two key insights: 💎 A plain transformer (e.g., vanilla DINO) is enough. No specialized architecture. ✨ A single depth-ray representation is enough. No complex 3D tasks. Three series of models have been released: the main DA3 series, a monocular metric estimation series, and a monocular depth estimation series. The core team members, aside from me: Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen. 👇(1/n) #DepthAnything3

Bingyi Kang

515,250 views • 8 months ago

In collaboration with Intel, our Depth Fusion showcases the power of our LDM3D diffusion model in generating 360° views from text prompts provided by the user. The LDM3D diffusion model generates a 2D RGB image and its corresponding relative depth map providing a complete RGBD representation corresponding to the text prompt. The LDM 3D model is a specialized version of the stable diffusion V 1.4 model that has been modified to fit both image and depth map data.The model was then fine tuned on a subset of the Laion400M data set - large scale image caption data set. The depth maps used to fine tune our model were generated by the DPTBeiT large 512 depth estimation model that provides highly accurate relative depth estimates for each pixel. We take the generated 2D RGB image and depth map and use them to compute a 360° projection using touchdesigner. Touchdesigner is a versatile platform that allows for the creation of immersive and interactive multimedia experiences. Our application harnesses the power of touchdesigner to bring the generated 360° views to life, providing users with a unique and engaging way to experience their text prompts, whether it’s a description of a tranquil forest, a noisy cityscape or a futuristic sci fi world. Our depth fusion can bring these concepts to life in a vivid and immersive detail. - Scottie Fox, VP Engineering Blockade Labs ScottieFox #AI #VR #3D #gamedev #stablediffusion
2:29

In collaboration with Intel, our Depth Fusion showcases the power of our LDM3D diffusion model in generating 360° views from text prompts provided by the user. The LDM3D diffusion model generates a 2D RGB image and its corresponding relative depth map providing a complete RGBD representation corresponding to the text prompt. The LDM 3D model is a specialized version of the stable diffusion V 1.4 model that has been modified to fit both image and depth map data.The model was then fine tuned on a subset of the Laion400M data set - large scale image caption data set. The depth maps used to fine tune our model were generated by the DPTBeiT large 512 depth estimation model that provides highly accurate relative depth estimates for each pixel. We take the generated 2D RGB image and depth map and use them to compute a 360° projection using touchdesigner. Touchdesigner is a versatile platform that allows for the creation of immersive and interactive multimedia experiences. Our application harnesses the power of touchdesigner to bring the generated 360° views to life, providing users with a unique and engaging way to experience their text prompts, whether it’s a description of a tranquil forest, a noisy cityscape or a futuristic sci fi world. Our depth fusion can bring these concepts to life in a vivid and immersive detail. - Scottie Fox, VP Engineering Blockade Labs ScottieFox #AI #VR #3D #gamedev #stablediffusion

Blockade Labs

11,439 views • 3 years ago

The robot’s “eyes” just received a big upgrade. LingBot-Depth 2.0, a depth-completion model with half the depth error just dropped. 12/16 benchmarks topped. Glass, mirrors, and transparent objects are so easy for us humans, but so hard for robots, because they do not behave like ordinary surfaces in a camera pipeline. A robot that misunderstands a balcony window or a table edge, will have a completely false planning inside a false world. Huge implecation. LingBot-Depth 2.0 takes an RGB image plus a broken depth map from a sensor and then outputs a cleaner depth map and a usable 3D point cloud. Numbers on LingBot-Depth 2.0 - Excels on glass, mirrors & transparent objects — where traditional depth cameras fail - Training data: 3M → 150M (50x scale-up) - 12 out of 16 first-place rankings on depth completion benchmarks - RMSE cut in half: 0.132 → 0.062 on the hardest indoor scenes LingBot-Vision trained on boundaries, because object edges carry the geometry robots need. No human boundary labels are used, which makes this approach easier to scale. The open-sourced LingBot-Vision is the general vision backbone, and LingBot-Depth 2.0 is the depth model built on it.
2:22

The robot’s “eyes” just received a big upgrade. LingBot-Depth 2.0, a depth-completion model with half the depth error just dropped. 12/16 benchmarks topped. Glass, mirrors, and transparent objects are so easy for us humans, but so hard for robots, because they do not behave like ordinary surfaces in a camera pipeline. A robot that misunderstands a balcony window or a table edge, will have a completely false planning inside a false world. Huge implecation. LingBot-Depth 2.0 takes an RGB image plus a broken depth map from a sensor and then outputs a cleaner depth map and a usable 3D point cloud. Numbers on LingBot-Depth 2.0 - Excels on glass, mirrors & transparent objects — where traditional depth cameras fail - Training data: 3M → 150M (50x scale-up) - 12 out of 16 first-place rankings on depth completion benchmarks - RMSE cut in half: 0.132 → 0.062 on the hardest indoor scenes LingBot-Vision trained on boundaries, because object edges carry the geometry robots need. No human boundary labels are used, which makes this approach easier to scale. The open-sourced LingBot-Vision is the general vision backbone, and LingBot-Depth 2.0 is the depth model built on it.

Rohan Paul

25,312 views • 1 month ago

Excited to introduce StereoPolicy, led by Evans Han. 📷📷🤖StereoPolicy is an effective way to add geometric cues to modern robot policy models while keeping the strengths of pretrained 2D encoders. ⁉️Why stereo for robot manipulation? Monocular RGB often lacks the depth cues needed for precise manipulation, while RGB-D and point clouds can be noisy or brittle, especially on reflective and transparent objects in real-world deployment. Instead of explicitly reconstructing disparity, depth, or point clouds, StereoPolicy directly fuses synchronized left/right RGB views to learn implicit stereo cues, avoiding extra reconstruction latency that can make real-time manipulation difficult. Project Page:
0:50

Excited to introduce StereoPolicy, led by Evans Han. 📷📷🤖StereoPolicy is an effective way to add geometric cues to modern robot policy models while keeping the strengths of pretrained 2D encoders. ⁉️Why stereo for robot manipulation? Monocular RGB often lacks the depth cues needed for precise manipulation, while RGB-D and point clouds can be noisy or brittle, especially on reflective and transparent objects in real-world deployment. Instead of explicitly reconstructing disparity, depth, or point clouds, StereoPolicy directly fuses synchronized left/right RGB views to learn implicit stereo cues, avoiding extra reconstruction latency that can make real-time manipulation difficult. Project Page:

Ruohan Zhang

850,714 views • 2 months ago

NVIDIA AI Open-Sources ViPE (Video Pose Engine): A Powerful and Versatile 3D Video Annotation Tool for Spatial AI ViPE integrates bundle adjustment with dense optical flow, sparse keypoint tracking, and metric depth priors to estimate camera intrinsics, poses, and dense depth maps at 3–5 FPS on a single GPU. It significantly improves over prior uncalibrated pose estimation methods, achieving 18% and 50% error reduction on TUM and KITTI benchmarks, respectively, and shows robustness to dynamic scenes and diverse camera models. Beyond the method, the NVIDIA team also released a large-scale dataset comprising ~100K real-world internet videos, 1M AI-generated videos, and 2K panoramic videos (≈96M frames) annotated with metric depth and poses. This dataset and engine aim to accelerate training for spatial AI tasks such as 3D reconstruction, video generation, and robotics.... full analysis: paper: codes: NVIDIA NVIDIA AI NVIDIAnewsroom NVIDIA Robotics NVIDIA AIDev NVIDIAdeveloper
1:13

NVIDIA AI Open-Sources ViPE (Video Pose Engine): A Powerful and Versatile 3D Video Annotation Tool for Spatial AI ViPE integrates bundle adjustment with dense optical flow, sparse keypoint tracking, and metric depth priors to estimate camera intrinsics, poses, and dense depth maps at 3–5 FPS on a single GPU. It significantly improves over prior uncalibrated pose estimation methods, achieving 18% and 50% error reduction on TUM and KITTI benchmarks, respectively, and shows robustness to dynamic scenes and diverse camera models. Beyond the method, the NVIDIA team also released a large-scale dataset comprising ~100K real-world internet videos, 1M AI-generated videos, and 2K panoramic videos (≈96M frames) annotated with metric depth and poses. This dataset and engine aim to accelerate training for spatial AI tasks such as 3D reconstruction, video generation, and robotics.... full analysis: paper: codes: NVIDIA NVIDIA AI NVIDIAnewsroom NVIDIA Robotics NVIDIA AIDev NVIDIAdeveloper

Marktechpost AI Dev News ⚡

217,453 views • 10 months ago

Are you tired of the low quality of iPhone lidar scans? I am! And that is why we are bringing this cutting-edge iPhone lidar scan enhancement function into production! With the guidance of normal and depth, the geometry can now reach the next level! Showcases: Please try our KIRI Engine 3.14 iOS version. Thanks to Xuqian and her AGSMesh paper ( which inspired us a lot, and also thanks to Haotong, Sida, and Jiaming for their stunning paper PromptDA ( which makes the depths way better. Of course, thanks to CJ and our intern team, Quanxiang and Ziteng, for helping with the development. #CVPR2025 #3DV2025 #GaussianSplatting #LiDAR
0:21

Are you tired of the low quality of iPhone lidar scans? I am! And that is why we are bringing this cutting-edge iPhone lidar scan enhancement function into production! With the guidance of normal and depth, the geometry can now reach the next level! Showcases: Please try our KIRI Engine 3.14 iOS version. Thanks to Xuqian and her AGSMesh paper ( which inspired us a lot, and also thanks to Haotong, Sida, and Jiaming for their stunning paper PromptDA ( which makes the depths way better. Of course, thanks to CJ and our intern team, Quanxiang and Ziteng, for helping with the development. #CVPR2025 #3DV2025 #GaussianSplatting #LiDAR

Chris make some 3D scans

105,512 views • 1 year ago

Standards are set to be met.....to guide, and shield us from certain mistakes, and irrational behaviors. As an individual, if you stand for nothing you fall for everything.....so your STANDARD is your WATCHWORD. Allow people stand on the business that pays them. Allow people wear the hair that fits them, the sunglasses that shield them, especially when they go out for "MEDICALS" As beauty is in the eyes of the beholder so is "DEPTH". In a entertainment reality show, where PERSONALITY is the focus, some have decided to mine "DEPTH OIL". Trust me all cards must be dealt, but assassinating the character of a player when you have equal access to play than be a paranoid commentator is crazy work. We went from; No depth Insecure Not all that Dumb girl syndrome And now Sugar baby So to the CEO DEPTH OIL LTD, we plead that you allow the NON-DEPTH breathe!!!!!🥹🥹🥹🥹 And it's NEPO BABY not SUGAR BABY. 🎥: Big Brother Naija MAIN CHARACTER DEDE #DedeAshiogwu #BBNaija
0:55

Standards are set to be met.....to guide, and shield us from certain mistakes, and irrational behaviors. As an individual, if you stand for nothing you fall for everything.....so your STANDARD is your WATCHWORD. Allow people stand on the business that pays them. Allow people wear the hair that fits them, the sunglasses that shield them, especially when they go out for "MEDICALS" As beauty is in the eyes of the beholder so is "DEPTH". In a entertainment reality show, where PERSONALITY is the focus, some have decided to mine "DEPTH OIL". Trust me all cards must be dealt, but assassinating the character of a player when you have equal access to play than be a paranoid commentator is crazy work. We went from; No depth Insecure Not all that Dumb girl syndrome And now Sugar baby So to the CEO DEPTH OIL LTD, we plead that you allow the NON-DEPTH breathe!!!!!🥹🥹🥹🥹 And it's NEPO BABY not SUGAR BABY. 🎥: Big Brother Naija MAIN CHARACTER DEDE #DedeAshiogwu #BBNaija

DEDE ASHIOGWU🌻

305,005 views • 1 year ago

We just tested what might be the most accurate motion transfer workflow we've seen so far. It's a little more involved than a standard workflow, but the results are pretty impressive. Here's the basic workflow: 1. Start with your reference footage. Create character and environment references. 2. Convert the reference footage into a depth map (with Depth-Anything tool on Fal: 3. (Optional) Layer on additional guidance like colorized character masks (with Meta's Segment Anything: and or skeleton poses (with 4. Feed the depth map into Seedance 2.0 Omni as your motion reference along with your character and environment references. 5. Generate your final video. After testing several variations, the depth map workflow consistently produced the cleanest and most accurate motion transfer. If you're trying to recreate specific performances or complex movement, this is one of the closest AI workflows we've seen to achieving production-quality results.
11:34

We just tested what might be the most accurate motion transfer workflow we've seen so far. It's a little more involved than a standard workflow, but the results are pretty impressive. Here's the basic workflow: 1. Start with your reference footage. Create character and environment references. 2. Convert the reference footage into a depth map (with Depth-Anything tool on Fal: 3. (Optional) Layer on additional guidance like colorized character masks (with Meta's Segment Anything: and or skeleton poses (with 4. Feed the depth map into Seedance 2.0 Omni as your motion reference along with your character and environment references. 5. Generate your final video. After testing several variations, the depth map workflow consistently produced the cleanest and most accurate motion transfer. If you're trying to recreate specific performances or complex movement, this is one of the closest AI workflows we've seen to achieving production-quality results.

Curious Refuge

29,932 views • 17 days ago

This is insane!🔥 This is a 720p (2D) video I shot of my kids 15 years ago! I used AI to upscale it, depth-map it, & used the depth map to convert it to 3D SBS, & then converted that to Spatial Video for Apple Vision Pro (MV-HEVC). My son is currently a senior in high school, and my daughter is a sophomore in college. This video has complete spatial depth when viewed in the Apple Vision Pro. Obviously, you can't appreciate that via this post. But it's like I'm there & I can see into this room. Talk about giving you the feels.🥹 This can be done with ANY video. 🤯 I'm working on refining the workflow, as it's a bit cumbersome. I plan to share it, but holy cow! I had to post this.
0:54

This is insane!🔥 This is a 720p (2D) video I shot of my kids 15 years ago! I used AI to upscale it, depth-map it, & used the depth map to convert it to 3D SBS, & then converted that to Spatial Video for Apple Vision Pro (MV-HEVC). My son is currently a senior in high school, and my daughter is a sophomore in college. This video has complete spatial depth when viewed in the Apple Vision Pro. Obviously, you can't appreciate that via this post. But it's like I'm there & I can see into this room. Talk about giving you the feels.🥹 This can be done with ANY video. 🤯 I'm working on refining the workflow, as it's a bit cumbersome. I plan to share it, but holy cow! I had to post this.

Blaine Brown

1,468,576 views • 2 years ago

Meta presents Sapiens Foundation for Human Vision Models discuss: We present Sapiens, a family of models for four fundamental human-centric vision tasks - 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction. Our models natively support 1K high-resolution inference and are extremely easy to adapt for individual tasks by simply fine-tuning models pretrained on over 300 million in-the-wild human images. We observe that, given the same computational budget, self-supervised pretraining on a curated dataset of human images significantly boosts the performance for a diverse set of human-centric tasks. The resulting models exhibit remarkable generalization to in-the-wild data, even when labeled data is scarce or entirely synthetic. Our simple model design also brings scalability - model performance across tasks improves as we scale the number of parameters from 0.3 to 2 billion. Sapiens consistently surpasses existing baselines across various human-centric benchmarks. We achieve significant improvements over the prior state-of-the-art on Humans-5K (pose) by 7.6 mAP, Humans-2K (part-seg) by 17.1 mIoU, Hi4D (depth) by 22.4% relative RMSE, and THuman2 (normal) by 53.5% relative angular error.
2:00

Meta presents Sapiens Foundation for Human Vision Models discuss: We present Sapiens, a family of models for four fundamental human-centric vision tasks - 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction. Our models natively support 1K high-resolution inference and are extremely easy to adapt for individual tasks by simply fine-tuning models pretrained on over 300 million in-the-wild human images. We observe that, given the same computational budget, self-supervised pretraining on a curated dataset of human images significantly boosts the performance for a diverse set of human-centric tasks. The resulting models exhibit remarkable generalization to in-the-wild data, even when labeled data is scarce or entirely synthetic. Our simple model design also brings scalability - model performance across tasks improves as we scale the number of parameters from 0.3 to 2 billion. Sapiens consistently surpasses existing baselines across various human-centric benchmarks. We achieve significant improvements over the prior state-of-the-art on Humans-5K (pose) by 7.6 mAP, Humans-2K (part-seg) by 17.1 mIoU, Hi4D (depth) by 22.4% relative RMSE, and THuman2 (normal) by 53.5% relative angular error.

AK

151,614 views • 1 year ago

Alibaba presents MIMO Controllable Character Video Synthesis with Spatial Decomposed Modeling Character video synthesis aims to produce realistic videos of animatable characters within lifelike scenes. As a fundamental problem in the computer vision and graphics community, 3D works typically require multi-view captures for per-case training, which severely limits their applicability of modeling arbitrary characters in a short time. Recent 2D methods break this limitation via pre-trained diffusion models, but they struggle for pose generality and scene interaction. To this end, we propose MIMO, a novel framework which can not only synthesize character videos with controllable attributes (i.e., character, motion and scene) provided by simple user inputs, but also simultaneously achieve advanced scalability to arbitrary characters, generality to novel 3D motions, and applicability to interactive real-world scenes in a unified framework. The core idea is to encode the 2D video to compact spatial codes, considering the inherent 3D nature of video occurrence. Concretely, we lift the 2D frame pixels into 3D using monocular depth estimators, and decompose the video clip to three spatial components (i.e., main human, underlying scene, and floating occlusion) in hierarchical layers based on the 3D depth. These components are further encoded to canonical identity code, structured motion code and full scene code, which are utilized as control signals of synthesis process. The design of spatial decomposed modeling enables flexible user control, complex motion expression, as well as 3D-aware synthesis for scene interactions. Experimental results demonstrate effectiveness and robustness of the proposed method.
1:29

Alibaba presents MIMO Controllable Character Video Synthesis with Spatial Decomposed Modeling Character video synthesis aims to produce realistic videos of animatable characters within lifelike scenes. As a fundamental problem in the computer vision and graphics community, 3D works typically require multi-view captures for per-case training, which severely limits their applicability of modeling arbitrary characters in a short time. Recent 2D methods break this limitation via pre-trained diffusion models, but they struggle for pose generality and scene interaction. To this end, we propose MIMO, a novel framework which can not only synthesize character videos with controllable attributes (i.e., character, motion and scene) provided by simple user inputs, but also simultaneously achieve advanced scalability to arbitrary characters, generality to novel 3D motions, and applicability to interactive real-world scenes in a unified framework. The core idea is to encode the 2D video to compact spatial codes, considering the inherent 3D nature of video occurrence. Concretely, we lift the 2D frame pixels into 3D using monocular depth estimators, and decompose the video clip to three spatial components (i.e., main human, underlying scene, and floating occlusion) in hierarchical layers based on the 3D depth. These components are further encoded to canonical identity code, structured motion code and full scene code, which are utilized as control signals of synthesis process. The design of spatial decomposed modeling enables flexible user control, complex motion expression, as well as 3D-aware synthesis for scene interactions. Experimental results demonstrate effectiveness and robustness of the proposed method.

AK

148,998 views • 1 year ago

🎙️ Charles Lee on LaMelo Ball's and Brandon Miller's minute restrictions: "We'll see how they recover after their first back-to-back in a while." "You just have to keep the process, the long-term vision, in mind even when you're in these moments. We want to win this game, obviously—every game we want to win really badly—and those two guys obviously help you do it." "But we do have really good depth, and so you lean on that depth in moments like this, and the maturity of those two guys to understand that there is a bigger picture and trying to maximize their availability. They did a good job of staying engaged even if they had to sit out for longer stretches than they're used to, and then when they came in the game, they had an impact."
1:52

🎙️ Charles Lee on LaMelo Ball's and Brandon Miller's minute restrictions: "We'll see how they recover after their first back-to-back in a while." "You just have to keep the process, the long-term vision, in mind even when you're in these moments. We want to win this game, obviously—every game we want to win really badly—and those two guys obviously help you do it." "But we do have really good depth, and so you lean on that depth in moments like this, and the maturity of those two guys to understand that there is a bigger picture and trying to maximize their availability. They did a good job of staying engaged even if they had to sit out for longer stretches than they're used to, and then when they came in the game, they had an impact."

r/CharlotteHornets

26,573 views • 8 months ago

With all due respect to my friend Doomer, the reason they didn’t use selective focus on the Rocky stair shot is because they couldn’t. The tech wasn’t there yet. Rocky was the first film the use a Steadicam. This was groundbreaking tech and changed cinematic language forever. That's the reason the stair scene is so iconic--because audiences had never seen anything like it before. If we were doing that shot today, we’d have a focus puller (called a 1st AC in the USA) manually focusing the lens on a wireless follow focus. Because there’s no other way to pull focus with a fast moving steadi cam following a fast moving subject. With a shallow depth of field Rocky would have been running in and out of focus. But back then they had no choice. So they used the largest depth of field they could get away with so that Rocky stayed in focus the entire time. If they’d had the tech to do selective focus they 100% would have used it--because cinema is all about selective focus. Steadicam inventor Garret Brown actually shot a test with his GF running on the same stairs used in Rocky (see video). And again, he was forced to use a deep depth of field because there was no way to accurately focus the camera while he was moving.
0:53

With all due respect to my friend Doomer, the reason they didn’t use selective focus on the Rocky stair shot is because they couldn’t. The tech wasn’t there yet. Rocky was the first film the use a Steadicam. This was groundbreaking tech and changed cinematic language forever. That's the reason the stair scene is so iconic--because audiences had never seen anything like it before. If we were doing that shot today, we’d have a focus puller (called a 1st AC in the USA) manually focusing the lens on a wireless follow focus. Because there’s no other way to pull focus with a fast moving steadi cam following a fast moving subject. With a shallow depth of field Rocky would have been running in and out of focus. But back then they had no choice. So they used the largest depth of field they could get away with so that Rocky stayed in focus the entire time. If they’d had the tech to do selective focus they 100% would have used it--because cinema is all about selective focus. Steadicam inventor Garret Brown actually shot a test with his GF running on the same stairs used in Rocky (see video). And again, he was forced to use a deep depth of field because there was no way to accurately focus the camera while he was moving.

Rambo Van Halen

20,578 views • 8 months ago

Enabling autonomous vehicles perceive their environment using only off-the-shelf cameras has been a long term research objective at Swaayatt Robots. This demo highlights the capabilities of our on-road perception system which is able to detect obstacles, road boundaries, lane markers in images, as well as compute depth of the complex scenes in the environment. The output shown in this video is end-to-end raw output from our deep learning system, without any post processing. The current system, with joint computation of obstacles, lane/road boundaries, and depth, works at 30 FPS on an embedded GPU in our autonomous vehicle, and can achieve higher FPS with further optimization -- which is currently a research in progress. This system is being scaled up for both the day and night operations, and we will showcase its strength towards enabling autonomous driving on a mountainous environment with unpaved roads, in the absence of any delimiters. #deeplearning #autonomousdriving #autonomousvehicles #machinelearning
3:27

Enabling autonomous vehicles perceive their environment using only off-the-shelf cameras has been a long term research objective at Swaayatt Robots. This demo highlights the capabilities of our on-road perception system which is able to detect obstacles, road boundaries, lane markers in images, as well as compute depth of the complex scenes in the environment. The output shown in this video is end-to-end raw output from our deep learning system, without any post processing. The current system, with joint computation of obstacles, lane/road boundaries, and depth, works at 30 FPS on an embedded GPU in our autonomous vehicle, and can achieve higher FPS with further optimization -- which is currently a research in progress. This system is being scaled up for both the day and night operations, and we will showcase its strength towards enabling autonomous driving on a mountainous environment with unpaved roads, in the absence of any delimiters. #deeplearning #autonomousdriving #autonomousvehicles #machinelearning

Sanjeev Sharma

18,478 views • 1 year ago

In a November 25, 2019 (ONE MONTH BEFORE COVID-19 BEGAN) interview of Dr. Anthony Fauci on GZERO World with Ian Bremmer, titled “Worried Sick | Anthony Fauci In-depth Interview.” “And then the other thing that I aspirationally hope to be able to encounter is the ability to rapidly respond to something brand new, whether it’s a brand new pandemic or as you mentioned, a brand new attack upon us deliberately by bioterror. That’s where we have a ways to go, but we’re making progress in that regard.” - Dr. Anthony Fauci #COVID19 #Fauci
0:26

In a November 25, 2019 (ONE MONTH BEFORE COVID-19 BEGAN) interview of Dr. Anthony Fauci on GZERO World with Ian Bremmer, titled “Worried Sick | Anthony Fauci In-depth Interview.” “And then the other thing that I aspirationally hope to be able to encounter is the ability to rapidly respond to something brand new, whether it’s a brand new pandemic or as you mentioned, a brand new attack upon us deliberately by bioterror. That’s where we have a ways to go, but we’re making progress in that regard.” - Dr. Anthony Fauci #COVID19 #Fauci

Nick D'Agostino

64,377 views • 9 days ago

So many couples spend years managing the logistics of life together without ever stopping to ask the deeper questions: What does our love mean? Who are we becoming together? And what do we want our next chapter to look like? On June 19 - 22, 2026, Esther and I are gathering couples in Santa Fe for a retreat named The Meaning of Us. Just to be clear, this is not a couple therapy, but an opportunity to: - Gain a shared vocabulary for what you each most deeply want - Understand your relationship as the complex, living thing it actually is - Do the individual work that makes the relational work possible - See your differences as a source of depth rather than friction We'd love to have you join us:
1:37

So many couples spend years managing the logistics of life together without ever stopping to ask the deeper questions: What does our love mean? Who are we becoming together? And what do we want our next chapter to look like? On June 19 - 22, 2026, Esther and I are gathering couples in Santa Fe for a retreat named The Meaning of Us. Just to be clear, this is not a couple therapy, but an opportunity to: - Gain a shared vocabulary for what you each most deeply want - Understand your relationship as the complex, living thing it actually is - Do the individual work that makes the relational work possible - See your differences as a source of depth rather than friction We'd love to have you join us:

Dr. Arthur Brooks

32,592 views • 2 months ago

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.
0:33

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.

AK

122,560 views • 3 years ago

For the past 3 months, I've been heads down building and extending our multimodal data capture rig for Physical AI training. What you're seeing here is a single synchronized capture from our system — RGB, depth (stereo IR camera) feed; real-time 3D hand reconstruction, and a 23×20 tactile pressure grid from sensor gloves, all aligned and timestamped together. Getting here took a lot of unglamorous iteration. We reworked the pipeline to deliver clean, gap-free depth at scale after identifying early depth frame drops caused by data throughput bottlenecks in our capture box. We built per-channel calibration models that convert raw resistance signals from our flex sensor gloves into accurate, drift-corrected force values — solving the creep behavior that made a steady grip falsely appear to fade over time. Even though we’ve stayed quiet on socials, word of mouth spread — our rigs have already been deployed in data factories, manufacturing lines, and homes across the US, Europe, China, and Southeast Asia by our customers and ourselves. In the next few days, I'll be sharing more about how we're tackling rights-cleared data capture from the hardware level, and our closed-loop data-model flywheel — using model evaluation results to identify gaps and guide what data our customers need. More soon, stay tuned…
0:26

For the past 3 months, I've been heads down building and extending our multimodal data capture rig for Physical AI training. What you're seeing here is a single synchronized capture from our system — RGB, depth (stereo IR camera) feed; real-time 3D hand reconstruction, and a 23×20 tactile pressure grid from sensor gloves, all aligned and timestamped together. Getting here took a lot of unglamorous iteration. We reworked the pipeline to deliver clean, gap-free depth at scale after identifying early depth frame drops caused by data throughput bottlenecks in our capture box. We built per-channel calibration models that convert raw resistance signals from our flex sensor gloves into accurate, drift-corrected force values — solving the creep behavior that made a steady grip falsely appear to fade over time. Even though we’ve stayed quiet on socials, word of mouth spread — our rigs have already been deployed in data factories, manufacturing lines, and homes across the US, Europe, China, and Southeast Asia by our customers and ourselves. In the next few days, I'll be sharing more about how we're tackling rights-cleared data capture from the hardware level, and our closed-loop data-model flywheel — using model evaluation results to identify gaps and guide what data our customers need. More soon, stay tuned…

Grace Zhang

54,115 views • 1 month ago

We are excited to share our #CORL2024 paper on learning quadrotor obstacle avoidance from the visual stream of a single #eventcamera! Trained entirely in simulation! We demonstrate obstacle avoidance both in the dark and in a forest up to 5m/s. PDF: Video: Project page: Event cameras are sensors that output per-pixel-level intensity changes at microsecond latency resolution; they feature nearly zero motion blur and high dynamic range but produce a very large volume of events under significant ego-motion and further lack a high-fidelity continuous-time sensor model in simulation, making direct #sim2real transfer not possible. By leveraging depth prediction as a pretext task, we pre-train a reactive obstacle avoidance policy with “approximated” simulated events and then fine-tune the perception component with limited events-and-depth real-world data. This technique bridges the sim2real gap for #eventcameras! As at the current state, there is no continuous-time sensor model for event cameras, we hope that this work can finally spur future research leveraging simulation for training event-vision-based policies to create faster, agile robots! Kudos to Anish Bhattacharya, @marcocannic, Vijay Kumar Nikolai Matni UZH Science University of Zurich UZH Space Hub UZH IfI European Research Council (ERC) GRASP Laboratory Penn Engineering
1:25

We are excited to share our #CORL2024 paper on learning quadrotor obstacle avoidance from the visual stream of a single #eventcamera! Trained entirely in simulation! We demonstrate obstacle avoidance both in the dark and in a forest up to 5m/s. PDF: Video: Project page: Event cameras are sensors that output per-pixel-level intensity changes at microsecond latency resolution; they feature nearly zero motion blur and high dynamic range but produce a very large volume of events under significant ego-motion and further lack a high-fidelity continuous-time sensor model in simulation, making direct #sim2real transfer not possible. By leveraging depth prediction as a pretext task, we pre-train a reactive obstacle avoidance policy with “approximated” simulated events and then fine-tune the perception component with limited events-and-depth real-world data. This technique bridges the sim2real gap for #eventcameras! As at the current state, there is no continuous-time sensor model for event cameras, we hope that this work can finally spur future research leveraging simulation for training event-vision-based policies to create faster, agile robots! Kudos to Anish Bhattacharya, @marcocannic, Vijay Kumar Nikolai Matni UZH Science University of Zurich UZH Space Hub UZH IfI European Research Council (ERC) GRASP Laboratory Penn Engineering

Davide Scaramuzza

17,219 views • 1 year ago