Загрузка видео...

Не удалось загрузить видео

На главную

🤔Applying Depth Estimation models directly to videos can result in inconsistency between frames. 💪Well, not anymore. 🔥ChronoDepth is a new approach to video depth estimation that focuses on achieving both accuracy within each frame and consistency across frames. 📜More👇

Gradio's profile picture

Gradio

56,455 subscribers

19,986 просмотров • 2 лет назад •via X (Twitter)

Наука и технологииНовости и политикаОбразование
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.
0:13

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

MrNeRF

27,428 просмотров • 1 год назад

Spent some time this week playing with depth estimation models. Monocular depth estimation has come a long way, but in terms of metric accuracy and geometric consistency, stereo depth still has it beat by a wide margin. DepthPro (mono) vs FoundationStereo (stereo)
0:14

Spent some time this week playing with depth estimation models. Monocular depth estimation has come a long way, but in terms of metric accuracy and geometric consistency, stereo depth still has it beat by a wide margin. DepthPro (mono) vs FoundationStereo (stereo)

FrostyFridge

27,774 просмотров • 1 год назад

Google just revealed an ABSOLUTE depth estimation model 🤯 As opposed to recent depth models (Marigold, PatchFusion) which aim for maximum details, DMD aims to estimate the ABSOLUTE depth (in meters) within the image More details below ⬇️⬇️
0:16

Google just revealed an ABSOLUTE depth estimation model 🤯 As opposed to recent depth models (Marigold, PatchFusion) which aim for maximum details, DMD aims to estimate the ABSOLUTE depth (in meters) within the image More details below ⬇️⬇️

Alex Carlier

199,532 просмотров • 2 лет назад

Video diffusion models are just overqualified depth estimators! Deterministic single-pass depth estimation based on WanV2.1. - SOTA 5.5 AbsRel on ScanNet - data-efficient than baselines; - no temporal flicker + infinite-length estimation w/ zero scale drift.
0:18

Video diffusion models are just overqualified depth estimators! Deterministic single-pass depth estimation based on WanV2.1. - SOTA 5.5 AbsRel on ScanNet - data-efficient than baselines; - no temporal flicker + infinite-length estimation w/ zero scale drift.

Wildminder

49,371 просмотров • 3 месяцев назад

KEEP can enhance video face super-resolution by maintaining consistency across frames! It uses Kalman filtering to improve facial details, working well on both synthetic and real-world videos. Links ⬇️
0:20

KEEP can enhance video face super-resolution by maintaining consistency across frames! It uses Kalman filtering to improve facial details, working well on both synthetic and real-world videos. Links ⬇️

Dreaming Tulpa 🥓👑

252,813 просмотров • 1 год назад

Discover the right 3D Geometric Foundation Model for your task—whether it’s stereo matching, multi-view depth estimation, video depth, pose estimation, semantic understanding, or novel view synthesis. Explore more insights in our #E3DBench #FoundationModel #3D #GaussianSplatting. Project Webpage:
0:14

Discover the right 3D Geometric Foundation Model for your task—whether it’s stereo matching, multi-view depth estimation, video depth, pose estimation, semantic understanding, or novel view synthesis. Explore more insights in our #E3DBench #FoundationModel #3D #GaussianSplatting. Project Webpage:

Zhiwen(Aaron) Fan

30,010 просмотров • 1 год назад

Introducing 🛹 RollingDepth 🛹 — a universal monocular depth estimator for arbitrarily long videos! Our paper, “Video Depth without Video Models,” delivers exactly that, setting new standards in temporal consistency. Check out more details in the thread 🧵
0:17

Introducing 🛹 RollingDepth 🛹 — a universal monocular depth estimator for arbitrarily long videos! Our paper, “Video Depth without Video Models,” delivers exactly that, setting new standards in temporal consistency. Check out more details in the thread 🧵

Anton Obukhov

49,830 просмотров • 1 год назад

I added a node to my custom node that converts the depth image to a normal map! This is useful for 2D-style images where Normals Estimation doesn't work well.
0:16

I added a node to my custom node that converts the depth image to a normal map! This is useful for 2D-style images where Normals Estimation doesn't work well.

toyxyz

117,354 просмотров • 1 год назад

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page: Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.
0:17

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page: Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.

AK

375,090 просмотров • 3 лет назад

Nano is a depth-aware atmospheric haze plugin that uses ML depth estimation to add physically accurate fog and light scattering to your footage. Works *best* on log footage with visible light sources - it analyzes scene highlights then creates airlight (atmospheric scatter) and halation (light bloom) that responds to actual depth in the scene. Pretty clever approach to getting that cinematic haze look without having to pump a fog machine on set. Makes the OG Trapcode Shine look extremely dated (basically 2D light streaks masked by luminance values), and is yet way more controllable than the current crop of generative AI video-to-video tools.
0:18

Nano is a depth-aware atmospheric haze plugin that uses ML depth estimation to add physically accurate fog and light scattering to your footage. Works *best* on log footage with visible light sources - it analyzes scene highlights then creates airlight (atmospheric scatter) and halation (light bloom) that responds to actual depth in the scene. Pretty clever approach to getting that cinematic haze look without having to pump a fog machine on set. Makes the OG Trapcode Shine look extremely dated (basically 2D light streaks masked by luminance values), and is yet way more controllable than the current crop of generative AI video-to-video tools.

Bilawal Sidhu

275,142 просмотров • 1 год назад

AnyDepth: Lightweight zero-shot monocular depth estimation; surpasses DPT; - nicely preserves detail.
0:17

AnyDepth: Lightweight zero-shot monocular depth estimation; surpasses DPT; - nicely preserves detail.

Wildminder

18,152 просмотров • 5 месяцев назад

Introducing Depth Estimation with NVIDIA’s FoundationStereo—a new open tutorial and codebase for metric depth sensing using Project Aria Gen 2 devices. ⚡️ Real-time ready (20 FPS) 🤖 Zero-shot generalization ⚡️Stereo image rectification and disparity estimation 🔓 Open Source Check out the code and tutorial: 🔌Chris Sweeney
0:11

Introducing Depth Estimation with NVIDIA’s FoundationStereo—a new open tutorial and codebase for metric depth sensing using Project Aria Gen 2 devices. ⚡️ Real-time ready (20 FPS) 🤖 Zero-shot generalization ⚡️Stereo image rectification and disparity estimation 🔓 Open Source Check out the code and tutorial: 🔌Chris Sweeney

Project Aria @Meta

32,949 просмотров • 1 месяц назад

How can we use test-time compute for spatial understanding? 🤔 In InterPose, we propose to repeatedly sample generative video models to help two-view pose estimation and reconstruction, by leveraging the video models' keyframe interpolation abilities. A 🧵... (1/8)
0:17

How can we use test-time compute for spatial understanding? 🤔 In InterPose, we propose to repeatedly sample generative video models to help two-view pose estimation and reconstruction, by leveraging the video models' keyframe interpolation abilities. A 🧵... (1/8)

Ricardo Martin-Brualla

21,685 просмотров • 1 год назад

Pika just dropped a fantastic feature (again) Now, Pika lets you have up to five frames, and you can prompt for EACH transition for up to a 25-second video. I have always wondered why we can't have more than start and end frames... Well, now you can (pun intended):
0:12

Pika just dropped a fantastic feature (again) Now, Pika lets you have up to five frames, and you can prompt for EACH transition for up to a 25-second video. I have always wondered why we can't have more than start and end frames... Well, now you can (pun intended):

Alex Patrascu

69,973 просмотров • 1 год назад

Google Gemini can analyze videos to create extra frames and shots. You can use this to create new images that are from different camera angles and styles. It does quite well to maintain the lighting and video scene as well. Example.
0:16

Google Gemini can analyze videos to create extra frames and shots. You can use this to create new images that are from different camera angles and styles. It does quite well to maintain the lighting and video scene as well. Example.

Jerrod Lew

28,102 просмотров • 1 год назад

wowwow Holy Shit I let Opus 4.7 pick any art on my computer they'd like to use as a starting frame for an animation, and then prompt gptimage2 for animation frames however they'd like to make a 100 frame animation here was the result note - gptimage2 only made 4 images, each one was a 5x5 grid of frames, so that totals to 100
0:12

wowwow Holy Shit I let Opus 4.7 pick any art on my computer they'd like to use as a starting frame for an animation, and then prompt gptimage2 for animation frames however they'd like to make a 100 frame animation here was the result note - gptimage2 only made 4 images, each one was a 5x5 grid of frames, so that totals to 100

w̸͕͂͂a̷͔̗͐t̴̙͗e̵̬̔̕r̴̰̓̊m̵͙͖̓̽a̵̢̗̓͒r̸̲̽ķ̷͔́͝

12,263 просмотров • 2 месяцев назад

Gave a robot 3D vision with just a regular camera👁️ Full Tutorial: Deployed #Depth Anything V3​ on NVIDIA Robotics Jetson AGX Orin. It estimates depth from 2D images in real-time—no special sensors needed. Just need #Monocular depth estimation + #TensorRT​ optimization + #ROS2​ integration. 👉 Learn more about reComputer Robotics J5011: #TheAIHardwarePartner
0:18

Gave a robot 3D vision with just a regular camera👁️ Full Tutorial: Deployed #Depth Anything V3​ on NVIDIA Robotics Jetson AGX Orin. It estimates depth from 2D images in real-time—no special sensors needed. Just need #Monocular depth estimation + #TensorRT​ optimization + #ROS2​ integration. 👉 Learn more about reComputer Robotics J5011: #TheAIHardwarePartner

Seeed Studio

16,966 просмотров • 4 месяцев назад

Distill-Any-Depth app is live on Hugging Face! 🔥🔥
0:12

Distill-Any-Depth app is live on Hugging Face! 🔥🔥

Gradio

48,874 просмотров • 1 год назад

🤯 𝐒𝐥𝐢𝐜𝐞𝐝𝐢𝐭 : A revolutionary approach to zero-shot video editing✏️🎥 using text-to-image diffusion models 🎞️Slicedit achieves great temporal consistency by leveraging spatio-temporal "slices"🔀
0:10

🤯 𝐒𝐥𝐢𝐜𝐞𝐝𝐢𝐭 : A revolutionary approach to zero-shot video editing✏️🎥 using text-to-image diffusion models 🎞️Slicedit achieves great temporal consistency by leveraging spatio-temporal "slices"🔀

Gradio

19,584 просмотров • 2 лет назад

Since Midjourney announced that they support creating videos with specific start and end frames, I had to try it. So I retextured the images with the JSON below and then used start and end frames to create a short video! Prompt 👇
0:11

Since Midjourney announced that they support creating videos with specific start and end frames, I had to try it. So I retextured the images with the JSON below and then used start and end frames to create a short video! Prompt 👇

Ege

326,982 просмотров • 11 месяцев назад