正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Introducing ⇆ Marigold-DC — our training-free zero-shot approach to monocular Depth Completion with guided diffusion! If you have ever wondered how else a long denoising diffusion schedule can be useful, we have an answer for you! Details 🧵

Anton Obukhov

2,667 subscribers

39,295 次观看 • 1 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,037 次观看 • 1 年前

Introducing 🛹 RollingDepth 🛹 — a universal monocular depth estimator for arbitrarily long videos! Our paper, “Video Depth without Video Models,” delivers exactly that, setting new standards in temporal consistency. Check out more details in the thread 🧵

Introducing 🛹 RollingDepth 🛹 — a universal monocular depth estimator for arbitrarily long videos! Our paper, “Video Depth without Video Models,” delivers exactly that, setting new standards in temporal consistency. Check out more details in the thread 🧵

Anton Obukhov

49,830 次观看 • 1 年前

Can you make a jigsaw puzzle with two different solutions? Or an image that changes appearance when flipped? We can do that, and a lot more, by using diffusion models to generate optical illusions! Continue reading for more illusions and method details 🧵

Can you make a jigsaw puzzle with two different solutions? Or an image that changes appearance when flipped? We can do that, and a lot more, by using diffusion models to generate optical illusions! Continue reading for more illusions and method details 🧵

Daniel Geng

125,781 次观看 • 2 年前

Humans are social creatures and physical interaction with other humans plays an important role in our everyday life. In this thread, we explain how such interactions can be reconstructed in 3D from a single image utilising a denoising diffusion probabilistic model. (1/n)

Humans are social creatures and physical interaction with other humans plays an important role in our everyday life. In this thread, we explain how such interactions can be reconstructed in 3D from a single image utilising a denoising diffusion probabilistic model. (1/n)

Lea Wilken/Müller

76,538 次观看 • 2 年前

High-resolution image and video generation is hitting a wall because attention in DiTs scales quadratically with token count. But does every pixel need to be in full resolution? Introducing Foveated Diffusion: a new approach for efficient diffusion-based generation that allocates compute where it matters most. 1/7🧵

High-resolution image and video generation is hitting a wall because attention in DiTs scales quadratically with token count. But does every pixel need to be in full resolution? Introducing Foveated Diffusion: a new approach for efficient diffusion-based generation that allocates compute where it matters most. 1/7🧵

Gordon Wetzstein

163,340 次观看 • 3 个月前

1/ Happy to share UniDisc - Unified Multimodal Discrete Diffusion – We train a 1.5 billion parameter transformer model from scratch on 250 million image/caption pairs using a **discrete diffusion objective**. Our model has all the benefits of diffusion models but now in multimodal space! - flexible compute-quality tradeoff, zero-shot inpainting and editing, better control via classifier-free guidance and lower latency! We open source everything - our code, weights and the training dataset.

1/ Happy to share UniDisc - Unified Multimodal Discrete Diffusion – We train a 1.5 billion parameter transformer model from scratch on 250 million image/caption pairs using a discrete diffusion objective. Our model has all the benefits of diffusion models but now in multimodal space! - flexible compute-quality tradeoff, zero-shot inpainting and editing, better control via classifier-free guidance and lower latency! We open source everything - our code, weights and the training dataset.

Mihir Prabhudesai

104,862 次观看 • 1 年前

You can build your own AI-apps powered by OpenAI and Stable Diffusion. **Even if you don't know how to code** Using no-code, build your own text and image generation apps with: - Bubble - OpenAI - Stable Diffusion Learn how:

You can build your own AI-apps powered by OpenAI and Stable Diffusion. Even if you don't know how to code Using no-code, build your own text and image generation apps with: - Bubble - OpenAI - Stable Diffusion Learn how:

Seth Kramer

6,893,693 次观看 • 3 年前

search algorithms are cool, but have you ever seen a maze being solved completely by a diffusion model?

search algorithms are cool, but have you ever seen a maze being solved completely by a diffusion model?

Rikard Radovac

20,663 次观看 • 1 年前

If you ever wondered how deer react to fireworks I found the answer in our field last night 😂

If you ever wondered how deer react to fireworks I found the answer in our field last night 😂

Sven Henrich

71,884 次观看 • 7 个月前

Learning bimanual, contact-rich robot manipulation policies that generalize over diverse objects has long been a challenge. Excited to share our work: Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation! 🧵1/n

Learning bimanual, contact-rich robot manipulation policies that generalize over diverse objects has long been a challenge. Excited to share our work: Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation! 🧵1/n

Xuanlin Li (Simon)

20,877 次观看 • 1 年前

Diffusions are excellent in creating fantastic images and videos 🔎 We cooked a *diffusion* model to synthesize structured data #ICLR2025 🔥 Introducing TabDiff, a mixed-type diffusion model for generating synthetic tabular data, imputing missing values, and beyond! 🧵 1/n

Diffusions are excellent in creating fantastic images and videos 🔎 We cooked a diffusion model to synthesize structured data #ICLR2025 🔥 Introducing TabDiff, a mixed-type diffusion model for generating synthetic tabular data, imputing missing values, and beyond! 🧵 1/n

Minkai Xu

50,434 次观看 • 1 年前

If you ever wondered what it’s like to be a producer working with an artist: 😂

If you ever wondered what it’s like to be a producer working with an artist: 😂

KIKI 🛖

38,680 次观看 • 1 年前

I made a tool called Diffusion Explorer that lets you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇

I made a tool called Diffusion Explorer that lets you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇

Alec Helbling

73,103 次观看 • 1 年前

Have you ever wondered how you get an F1 car onto a yacht before? 🛥️ #McLaren | #M7AReborn | @OKX

Have you ever wondered how you get an F1 car onto a yacht before? 🛥️ #McLaren | #M7AReborn | @OKX

McLaren Mastercard Formula 1 Team

6,552,309 次观看 • 1 年前

Chop the gradients ✂️! We found that truncating decoder gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.

Chop the gradients ✂️! We found that truncating decoder gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.

Felix Heide

28,323 次观看 • 2 个月前

Have you ever wondered how we transfer print books to HeinOnline? Behold: the book scanner😲

Have you ever wondered how we transfer print books to HeinOnline? Behold: the book scanner😲

HeinOnline

102,058 次观看 • 3 年前

AnyDepth: Lightweight zero-shot monocular depth estimation; surpasses DPT; - nicely preserves detail.

AnyDepth: Lightweight zero-shot monocular depth estimation; surpasses DPT; - nicely preserves detail.

Wildminder

18,152 次观看 • 5 个月前

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page: Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page: Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.

AK

375,090 次观看 • 3 年前

Have you ever wondered if your screwing right

Sensitive content

Have you ever wondered if your screwing right

Lorelei Finds

51,676 次观看 • 1 年前

🤯 𝐒𝐥𝐢𝐜𝐞𝐝𝐢𝐭 : A revolutionary approach to zero-shot video editing✏️🎥 using text-to-image diffusion models 🎞️Slicedit achieves great temporal consistency by leveraging spatio-temporal "slices"🔀

🤯 𝐒𝐥𝐢𝐜𝐞𝐝𝐢𝐭 : A revolutionary approach to zero-shot video editing✏️🎥 using text-to-image diffusion models 🎞️Slicedit achieves great temporal consistency by leveraging spatio-temporal "slices"🔀

Gradio

19,584 次观看 • 2 年前