✨UPDATE | ControlVideo demo can now load custom diffusion... models stored in Hugging Face diffusers gallery. —› Here's an example with the Ghibli Diffusion from Nitrosocke — demo:show more

GANs are so back?! Scientists from Brown and Cornell... have published a paper with a ✨ modern architecture GAN ✨ that is 🗿 stable to train 🗿 and competitive with SOTA GANs and even diffusion models Paper and demo 👇show more

New ML gem on the hub: LDM3D by Intel.... This diffusion model generates image & depth from text prompts. Using a custom @gradio 6dof three.js component you can generate immersive 360-degree views from prompts demo: model:show more

Diffusion Transformers aren't just generative models, but also powerful... multi-modal encoders. ConceptAttention creates rich heatmaps of text concepts in images from DiT representations. This even works on real images, and can be applied to tasks like segmentation! Demo 👇show more

The latent space of earlier generative models like GANS... can linearly encode concepts of the data. What if the data was model weights? We present weights2weights, a subspace in diffusion weights that behaves as an interpretable latent space over customized diffusion models.show more

Excited to introduce Diffusion Augmented Agents (DAAGs)✨. We give... an agent control of a diffusion model, so it can create its own *synthetic experience*.🪄 The result is a lifelong agent that can learn new reward detectors and policies, much more efficiently. Here's how. 👇show more

We have released the code and weights for our... #CVPR2023 paper "Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model"! code: abs: project: The demo is below:show more

Here's what you can expect in Portal Fantasy. Our... demo is live now! Play the demo and wishlist on Steam: #SteamDemo #SteamWishlist #PortalFantasyshow more

RF-DETR is now available in Hugging Face transformers state... of the art in both detection and segmentation, outperforming YOLO architectures - checkpoints: - demo: - docs:show more

Massive update for AI Engineers! Training diffusion models just... got a lot easier. dLLM is an open-source library that does for diffusion models what Hugging Face did for transformers. Here's why this matters: Traditional autoregressive models generate text left-to-right, one token at a time. Diffusion models work differently - they refine the entire sequence iteratively, giving you better control over generation quality and more flexible editing capabilities. The problem? Building and training these models required stitching together scattered tools and reimplementing research papers from scratch. Most of the tooling has been scattered and is hard to reproduce. dLLM changes this: It unifies everything you need to train, evaluate, and deploy diffusion language models: ↳ Scalable training with LoRA, DeepSpeed, and FSDP support ↳ Unified evaluation that abstracts away inference complexity ↳ Ready-to-use recipes for pretraining, finetuning, and evaluation The library includes implementations of models like LLaDA and Dream, plus training algorithms like Edit Flows that enable insertion, deletion, and substitution operations. The team just released ModernBERT-Chat models showing you can turn BERT into lightweight chatbots through masked instruction tuning. This is practical and worth exploring. The setup is straightforward. Run locally with Accelerate or scale to multi-node clusters with Slurm. If you're working with language models and want to explore diffusion-based approaches without rebuilding infrastructure, dLLM gives you a production-ready starting point. Link to the repo in the next tweet.show more

Stable Diffusion 100% in the browser with running at... < 1s per image ⚡️ Thanks to SD Turbo by Axel Sauer & al Stability AI and diffusers.js on webGPU by Arthur Islamov Demo: More info in ↓🧵show more

Hugging Face 🤝 Jan You can now use Hugging... Face as a remote model provider in Jan. Go to Settings -> Model Providers -> add your Hugging Face API key. Then open a new chat and pick a model from Hugging Face. Works with any model in Hugging Face in Jan.show more

Chop the gradients ✂️! We found that truncating decoder... gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.show more

The Final Fantasy VII Rebirth demo update is out... now! Experience more of the game in "Dawn of a New Era", set in an open area near Junon which includes content unique to the demo. The update also improves visual fidelity in Performance Mode. #FF7Rshow more

DimensionX: Create Any 3D and 4D Scenes from a... Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.show more

Introducing DI-PCG: A diffusion-based Image2PCG framework that automates Procedural... Content Generation, and our first step towards neural symbolic 3D generation. The demo showcases six object categories and can be extended to support customized PCGs. Links in comments. CC: Wang Zhaoshow more

Impeccable 3 natively supports crafting new design with diffusion... (e.g. GPT Image 2) in harnesses that support it (e.g. Codex, Gemini w/ nano banana plugin). Impeccable now uses the best mode of operation for net-new vs iterative design: 1) completely new design: diffusion-first, then build w/ best practices for brand or product design (new in 3.0) 2) iterative design (additions/changes): design-system first, discovers your components, reads DESIGN/PRODUCT.md, goes to work. new live mode (new in 3.0) makes it extra convenient. Here's a tutorial documenting the whole workflow (incl. how I made the demo site in the video):show more

Today, we are adding Stable Video Diffusion, our foundation... model for generative video to the Stability AI Developer Platform API. The model can generate 2 seconds of video, comprising of 25 generated frames and 24 frames of FILM interpolation, within an average time of 41 seconds. Developers interested in utilizing Stable Video Diffusion through an API can access it now on the Stability AI Developer Platform. Learn more here:show more

The Duskfade demo is now live on Xbox Series... X/S! ✨ ⚔️ 3D action platformer with fluid combat 🌙 Over an hour of gameplay in the demo 🕰️ Explore a vibrant world packed with secrets 🐦 Meet your sidekick, Cuckoo! Thanks for all the support! #XBOXshow more

We are also releasing self-contained lecture notes that explain... flow matching and diffusion models from scratch. This goes from "zero" to the state-of-the-art in modern Generative AI. 📖 Read the notes here: Joint work with Ezra Erives.show more