🚨New paper! Generative models are often “miscalibrated”. We calibrate... diffusion models, LLMs, and more to meet desired distributional properties. E.g. we finetune protein models to better match the diversity of natural proteins.show more

Diffusion models are great, but we can squeeze out... so much more from them. The only problem is that it usually requires extra training or manual representation editing. In our new paper, we show that with the current capabilities of LLMs, it is much simpler than we thought!show more

Diffusion models have amazing image creation abilities. But how... well does their generative knowledge transfer to discriminative tasks? We present Diffusion Classifier: strong classification results with pretrained conditional diffusion models, *with no additional training*! 1/9show more

Diffusion models are sensitive to small changes in the... input noise. We introduce Alias-Free Latent Diffusion Models (𝗔𝗙-𝗟𝗗𝗠) at #CVPR2025. It achieves shift-equivariance and generates consistent outputs. Project: arXiv:show more

The latent space of earlier generative models like GANS... can linearly encode concepts of the data. What if the data was model weights? We present weights2weights, a subspace in diffusion weights that behaves as an interpretable latent space over customized diffusion models.show more

Language models understand natural proteins. But can they generalize... beyond, to design completely new proteins from scratch? New preprint: A 🧵show more

Generative models can’t discover what they can’t reach. We’re... excited to introduce ActFlow: a continued pre-training scheme that actively expands the valid design space reachable by flow and diffusion models. We call this generable set expansion — a new learning principle for out-of-distribution generative modeling, and a step toward evolvable search spaces for scientific discovery. (1/5)show more

We are also releasing self-contained lecture notes that explain... flow matching and diffusion models from scratch. This goes from "zero" to the state-of-the-art in modern Generative AI. 📖 Read the notes here: Joint work with Ezra Erives.show more

Major AI breakthrough: Diffusion Large Language Models are here!... They're 10x faster and 10x cheaper than traditional LLMs. Here's everything you need to know:show more

1/ Happy to share VADER: Video Diffusion Alignment via... Reward Gradients. We adapt foundational video diffusion models using pre-trained reward models to generate high-quality, aligned videos for various end-applications. Below we generated a short movie using VADER 😀, we used ChatGPT to write a script and an off-the-shelf AI music generator to generate the sound. Our code & weights are open-sourced:show more

We found a new way to get language models... to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.show more

Ever wondered how training dynamics differ between LLMs 🖋️... and Vision 👁️ models? We explore this and close the gap between VMs and LLMs in our #NeurIPS2024 paper "TrAct: Making First-layer Pre-Activations Trainable". Paper📜 Video🎥show more

Can we use motion to prompt diffusion models? Our... #ICLR2024 paper does just that. We propose Motion Guidance, a technique that allows users to edit an image by specifying “where things should move.”show more

New research from BlackForestLabsAI - Unofficial 🥳 Meet Self-Flow:... our self-supervised framework for image, audio, video & world models 🤖 Do generative models really need DINO to learn strong representations? We propose teaching them directly via a joint framework instead 🧵show more

It's Christmas morning: OpenAI and Anthropic shipped new models... on the same day! We tested GPT 5.3 Codex vs. Opus 4.6 head-to-head. Verdict: the models are converging. Here’s what we found 🧵show more

1. Stability AI launches StableLM The creators of Stable... Diffusion, Stability AI, just released a suite of open-sourced large language models (LLMs) called StableLM. This comes just 5 days after the public release of their text-to-image generative AI model, SDXL.show more

How can we use test-time compute for spatial understanding?... 🤔 In InterPose, we propose to repeatedly sample generative video models to help two-view pose estimation and reconstruction, by leveraging the video models' keyframe interpolation abilities. A 🧵... (1/8)show more

A major question in multimodal modeling is how to... leverage strong pre-trained models, such as industry-scale LLMs and VLMs, during training to avoid starting from scratch. This is important as the demand for rich and domain-specific multimodal models continues to increase, while training them from scratch is obviously impractical due to limited data and compute. The so-called “any-to-any” multimodal models can model a large diverse dictionary of modalities. That’s good. Their downside is that their architectures are often not decoder-only, which has limited their performance in practice and prevents them from leveraging strong pre-trained decoder-only models as priors. We are releasing MODUS (ICML Conference '26) a decoder-only any-to-any multimodal model to address some of these questions. A single transformer decoder predicts any modality from any others with no modality-specific heads, losses, or task pipelines. We show efficient adaptation of established models, e.g., BAGEL, to rich any-to-any multimodal modeling. We are releasing 14B to 77B parameter models. All materials are open-source. The download links and demos here 🧵show more

Switti -- a new scale-wise transformer for text-to-image generation... 🦾 🔥 Improved generation of fine-grained details. Outperforms existing T2I AR models and competes with state-of-the-art T2I diffusion models while being up to 7x faster.show more

My first PhD paper!🎉We learn *diffusion* models for code... generation that learn to directly *edit* syntax trees of programs. The result is a system that can incrementally write code, see the execution output, and debug it. 🧵1/nshow more

Harness the power of generative AI without just prompting... and hoping for the best. From Unbound Slop to Tripo🔜SIGGRAPH2026 to Unreal Engine, we love this workflow for generating 3D models.show more