Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Introducing ⇆ Marigold-DC — our training-free zero-shot approach to monocular Depth Completion with guided diffusion! If you have ever wondered how else a long denoising diffusion schedule can be useful, we have an answer for you! Details 🧵

Anton Obukhov

2,696 subscribers

39,324 views • 1 year ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,052 views • 1 year ago

Can you make a jigsaw puzzle with two different solutions? Or an image that changes appearance when flipped? We can do that, and a lot more, by using diffusion models to generate optical illusions! Continue reading for more illusions and method details 🧵

Can you make a jigsaw puzzle with two different solutions? Or an image that changes appearance when flipped? We can do that, and a lot more, by using diffusion models to generate optical illusions! Continue reading for more illusions and method details 🧵

Daniel Geng

125,878 views • 2 years ago

Humans are social creatures and physical interaction with other humans plays an important role in our everyday life. In this thread, we explain how such interactions can be reconstructed in 3D from a single image utilising a denoising diffusion probabilistic model. (1/n)

Humans are social creatures and physical interaction with other humans plays an important role in our everyday life. In this thread, we explain how such interactions can be reconstructed in 3D from a single image utilising a denoising diffusion probabilistic model. (1/n)

Lea Wilken/Müller

76,563 views • 2 years ago

Chop the gradients ✂️! We found that truncating decoder gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.

Chop the gradients ✂️! We found that truncating decoder gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.

Felix Heide

28,399 views • 3 months ago

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page: Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page: Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.

AK

375,160 views • 3 years ago

🧵1/5 Many of you have been asking for "the metro" details... We don’t want to spoil everything but for now all we can say is collection supply will be 2024. 1024 for community, 1024 for other on-chain/art focused communities. It will be free to mint!

🧵1/5 Many of you have been asking for "the metro" details... We don’t want to spoil everything but for now all we can say is collection supply will be 2024. 1024 for community, 1024 for other on-chain/art focused communities. It will be free to mint!

int art

29,452 views • 3 years ago

You asked (and asked, and asked… and asked) and finally, we have an answer: for the first time ever, we designed an entire Jet Lag format that you can buy and play with friends, anywhere in the world:

You asked (and asked, and asked… and asked) and finally, we have an answer: for the first time ever, we designed an entire Jet Lag format that you can buy and play with friends, anywhere in the world:

Sam (from Wendover)

346,014 views • 1 year ago

👤: would it be accurate to say you have zero desire to be a solo popstar? 🐯🐻: if i have zero desire, shouldn’t i stop being an artist? TAEHYUNG YOURE SO ICONIC FOR THIS RESPONSE

👤: would it be accurate to say you have zero desire to be a solo popstar? 🐯🐻: if i have zero desire, shouldn’t i stop being an artist? TAEHYUNG YOURE SO ICONIC FOR THIS RESPONSE

Carolyne🌱⁷⁼¹

201,276 views • 3 months ago

🐈‍⬛ : how about... we make a promise to be together forever? do you guys have a confident to keep that promise? "YESSSS" 🐈‍⬛ : then, i'll believe in you guys, and lets be together forever 🦊 : if you stay with us, we can be forever

🐈‍⬛ : how about... we make a promise to be together forever? do you guys have a confident to keep that promise? "YESSSS" 🐈‍⬛ : then, i'll believe in you guys, and lets be together forever 🦊 : if you stay with us, we can be forever

wyn

12,442 views • 6 months ago

"The disease that creeps into your program is called entitlement. If you can stomp it out with leadership, you can stay hungry. We have a saying around our place: we eat off the floor. If you're willing to eat off the floor, you can be special."

"The disease that creeps into your program is called entitlement. If you can stomp it out with leadership, you can stay hungry. We have a saying around our place: we eat off the floor. If you're willing to eat off the floor, you can be special."

The Winning Difference

219,195 views • 2 years ago

Serbia 2016, Before we can sign you, you have to run in the snow for an hour so we can be sure you are fit and you dont have any injury for a week …. Throwback to using your last life to survive 😂😂😂

Serbia 2016, Before we can sign you, you have to run in the snow for an hour so we can be sure you are fit and you dont have any injury for a week …. Throwback to using your last life to survive 😂😂😂

ABU AZEEZ

29,557 views • 1 year ago

“Are you alone, Mr. Leech? I know how that feels. We can be best friends, and we never have to be alone ever again” just end it all for me

“Are you alone, Mr. Leech? I know how that feels. We can be best friends, and we never have to be alone ever again” just end it all for me

matthew

23,060 views • 1 year ago

World modeling and imitation learning have largely been considered two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with *decoupled* diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -

World modeling and imitation learning have largely been considered two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with decoupled diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -

Abhishek Gupta

11,430 views • 1 year ago

"How long should I rest between sets?" Until you can hit the same weight for the same reps with the same quality. That is the answer. That has always been the answer. The answer is not 60 seconds. The answer is not 90 seconds. The answer is not whatever the timer on your watch app says. If you are doing heavy compounds, that might be 3 to 5 minutes. Sometimes longer. The exercise does not care about your podcast schedule. It needs the central nervous system to be ready and the muscles to be loaded again. Rushing rest periods is one of the great unforced errors in training. You sacrifice load, sacrifice reps, sacrifice tension, and convince yourself you have done a hard session because you sweated. You have done a cardio session with weights. They are not the same. Sit down. Breathe. Let the heart settle. Then lift the next set heavy enough to justify the wait.

"How long should I rest between sets?" Until you can hit the same weight for the same reps with the same quality. That is the answer. That has always been the answer. The answer is not 60 seconds. The answer is not 90 seconds. The answer is not whatever the timer on your watch app says. If you are doing heavy compounds, that might be 3 to 5 minutes. Sometimes longer. The exercise does not care about your podcast schedule. It needs the central nervous system to be ready and the muscles to be loaded again. Rushing rest periods is one of the great unforced errors in training. You sacrifice load, sacrifice reps, sacrifice tension, and convince yourself you have done a hard session because you sweated. You have done a cardio session with weights. They are not the same. Sit down. Breathe. Let the heart settle. Then lift the next set heavy enough to justify the wait.

Sama Hoole

26,463 views • 3 months ago

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:

Chenfeng_X

28,306 views • 1 year ago

May God not put our food in a hard place. If you don’t have to sell your soul for crumbs like this before you and your family can eat, you have no idea how God has been kind to you.

May God not put our food in a hard place. If you don’t have to sell your soul for crumbs like this before you and your family can eat, you have no idea how God has been kind to you.

OurFaveOnlineDoc 🇬🇧 🇳🇬

119,670 views • 6 months ago

You can't walk on our footpaths or drive on our roads ! We have a lot to be grateful for to our Govt Every day is an adventure !

You can't walk on our footpaths or drive on our roads ! We have a lot to be grateful for to our Govt Every day is an adventure !

Dr Aniruddha Malpani, MD

106,831 views • 1 year ago

Thank you from Caleb for everything you all have done for our community. With all your help the last three years, we have gathered almost 40,000 pounds of food during the Holidays, with more to be counted. You can still donate here:

Thank you from Caleb for everything you all have done for our community. With all your help the last three years, we have gathered almost 40,000 pounds of food during the Holidays, with more to be counted. You can still donate here:

Purdue Men's Basketball

25,075 views • 1 year ago