Introducing ⇆ Marigold-DC — our training-free zero-shot approach to... monocular Depth Completion with guided diffusion! If you have ever wondered how else a long denoising diffusion schedule can be useful, we have an answer for you! Details 🧵show more

DimensionX: Create Any 3D and 4D Scenes from a... Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.show more

Introducing 🛹 RollingDepth 🛹 — a universal monocular depth... estimator for arbitrarily long videos! Our paper, “Video Depth without Video Models,” delivers exactly that, setting new standards in temporal consistency. Check out more details in the thread 🧵show more

Can you make a jigsaw puzzle with two different... solutions? Or an image that changes appearance when flipped? We can do that, and a lot more, by using diffusion models to generate optical illusions! Continue reading for more illusions and method details 🧵show more

Humans are social creatures and physical interaction with other... humans plays an important role in our everyday life. In this thread, we explain how such interactions can be reconstructed in 3D from a single image utilising a denoising diffusion probabilistic model. (1/n)show more

High-resolution image and video generation is hitting a wall... because attention in DiTs scales quadratically with token count. But does every pixel need to be in full resolution? Introducing Foveated Diffusion: a new approach for efficient diffusion-based generation that allocates compute where it matters most. 1/7🧵show more

1/ Happy to share UniDisc - Unified Multimodal Discrete... Diffusion – We train a 1.5 billion parameter transformer model from scratch on 250 million image/caption pairs using a **discrete diffusion objective**. Our model has all the benefits of diffusion models but now in multimodal space! - flexible compute-quality tradeoff, zero-shot inpainting and editing, better control via classifier-free guidance and lower latency! We open source everything - our code, weights and the training dataset.show more

You can build your own AI-apps powered by OpenAI... and Stable Diffusion. **Even if you don't know how to code** Using no-code, build your own text and image generation apps with: - Bubble - OpenAI - Stable Diffusion Learn how:show more

search algorithms are cool, but have you ever seen... a maze being solved completely by a diffusion model?show more

If you ever wondered how deer react to fireworks... I found the answer in our field last night 😂show more

Learning bimanual, contact-rich robot manipulation policies that generalize over... diverse objects has long been a challenge. Excited to share our work: Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation! 🧵1/nshow more

Diffusions are excellent in creating fantastic images and videos... 🔎 We cooked a *diffusion* model to synthesize structured data #ICLR2025 🔥 Introducing TabDiff, a mixed-type diffusion model for generating synthetic tabular data, imputing missing values, and beyond! 🧵 1/nshow more

If you ever wondered what it’s like to be... a producer working with an artist: 😂show more

I made a tool called Diffusion Explorer that lets... you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇show more

Have you ever wondered how you get an F1... car onto a yacht before? 🛥️ #McLaren | #M7AReborn | @OKXshow more

Chop the gradients ✂️! We found that truncating decoder... gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.show more

Have you ever wondered how we transfer print books... to HeinOnline? Behold: the book scanner😲show more

AnyDepth: Lightweight zero-shot monocular depth estimation; surpasses DPT; -... nicely preserves detail.show more

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page:... Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.show more

🤯 𝐒𝐥𝐢𝐜𝐞𝐝𝐢𝐭 : A revolutionary approach to zero-shot video... editing✏️🎥 using text-to-image diffusion models 🎞️Slicedit achieves great temporal consistency by leveraging spatio-temporal "slices"🔀show more

Have you ever wondered if your screwing right