⚡ Video diffusion is slow. PDD changes that. 🚀... Introducing Parallel Decoding Distillation, our latest work on accelerating video diffusion models. Paper: Page: By Neta Shaul, Julius Berner, and Chao Liu. Below PDD on LTX-2.3 🔊show more

🚨Announcing our #ICLR2025 Oral! 🔥Diffusion LMs are on the... rise for parallel text generation! But unlike autoregressive LMs, they struggle with quality, fixed-length constraints & lack of KV caching. 🚀Introducing Block Diffusion—combining autoregressive and diffusion models for the best of both worlds! 👇1/7show more

Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models... "Block diffusion sequentially generates blocks of tokens by performing diffusion within each block and conditioning on previous blocks. By combining strength from autoregressive and diffusion models, block diffusion overcomes the limitations of both approaches by supporting variable-length, higher-quality generation and improving inference efficiency with KV caching and parallel sampling."show more

Warning: Dropping attention on video diffusion models may cause nightmares

Introducing Manifold Diffusion Fields (MDF), our new work on... learning generative models over fields defined on curved geometries. This is joint work with our intern Ahmed Elhag (who hasn’t even started his PhD yet!) and @jsusskin at Apple MLR 🧵show more

MMaDA-Parallel Multimodal Large Diffusion Language Models for Thinking-Aware Editing... and Generationshow more

🚀 Sol Video Inference Engine is here! An agent-native,... training-free full-stack accelerator for video diffusion. It auto-tunes cache + sparse attn + token pruning + quant + kernel fusion for any model/hardware/config. >2× end-to-end speedup on 64B Cosmos3-Super, 22B LTX-2.3 and 2B SANA-Video — near-lossless VBench quality, minimal human effort. Practical acceleration for real video gen deployment. 📄 Paper: 🌐 Project: 💻 Code: Proud of the team! 🎉show more

Introducing Diffusion Forcing, which unifies next-token prediction (eg LLMs)... and full-seq. diffusion (eg SORA)! It offers improved performance & new sampling strategies in vision and robotics, such as stable, infinite video generation, better diffusion planning, and more! (1/8)show more

NVIDIA just released a very impressive text-to-video paper. Video... Latent Diffusion Models (Video LDMs) use a diffusion model in a compressed latent space to generate high-resolution videos. Here's a brief overview of how it works: 1. Pre-train image LDM on a dataset of images. 2. Turn the image LDM into a Video LDM by adding temporal layers to model video frames. 3. Fine-tune the Video LDM on encoded video sequences to create a video generator. 4. Temporally align diffusion model upsamplers to generate high-resolution videos. 5. Validate Video LDM on real driving videos of 512x1024 resolution, achieving state-of-the-art performance. 6. Apply the approach in creative content creation with text-to-video modeling. Paper: Project:show more

Diffusion models are sensitive to small changes in the... input noise. We introduce Alias-Free Latent Diffusion Models (𝗔𝗙-𝗟𝗗𝗠) at #CVPR2025. It achieves shift-equivariance and generates consistent outputs. Project: arXiv:show more

Google is making progress on their diffusion models... It's... now as good as a Gemini 2.0 Flash Lite. The writing is on the wall, a majority of language AI use in the future will be diffusion models.show more

(1/2) Check out "𝐏𝐨𝐥𝐲𝐃𝐢𝐟𝐟: Generating 3D Polygonal Meshes with... Diffusion Models"! Our model operates directly on the polygons of 3D meshes and generates novel shapes as output through an iterative diffusion process.show more

This LTX 2.3 Video Outpainting LoRA by Mohamed Oumoumad... is no joke. So cool!!!!show more

SANA-Sprint One-Step Diffusion with Continuous-Time Consistency Distillation

DimensionX: Create Any 3D and 4D Scenes from a... Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.show more

Can we use motion to prompt diffusion models? Our... #ICLR2024 paper does just that. We propose Motion Guidance, a technique that allows users to edit an image by specifying “where things should move.”show more

DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion abs: project... page: github:show more

Today, we are releasing Stable Video Diffusion, our first... foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:show more

🚀 Introducing GenLit – Reformulating Single-Image Relighting as Video... Generation! We leverage video diffusion models to perform realistic near-field relighting from just a single image—No explicit 3D reconstruction or ray tracing required! No intermediate graphics buffers, directly in the pixel space! 📄 Dive into the paper: 🎥 Project page & demos: 🛠 Code coming soon! #GenerativeAI #ComputerVision #Relighting #DiffusionModels #Graphics 🧵 1/5show more

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page:... Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.show more