The Dream 7B (diffusion reasoning language model) is OUT!... 🚨 I built a demo so you can test it out (and check the diffusion process live) 𖣯🔍show more

LLaDA (the first Large Language Diffusion Model) is *just*... out 💥 and I've built a demo, try out now 👨‍💻 It's mesmerizing to watch the diffusion process 🌀, and it being a diffusion model gives you superpowers like "the 4th word has to be pineapple" 🦸 Demo and weights 👇show more

(1/2) Check out "𝐏𝐨𝐥𝐲𝐃𝐢𝐟𝐟: Generating 3D Polygonal Meshes with... Diffusion Models"! Our model operates directly on the polygons of 3D meshes and generates novel shapes as output through an iterative diffusion process.show more

I made a tool called Diffusion Explorer that lets... you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇show more

Excited to introduce Diffusion Augmented Agents (DAAGs)✨. We give... an agent control of a diffusion model, so it can create its own *synthetic experience*.🪄 The result is a lifelong agent that can learn new reward detectors and policies, much more efficiently. Here's how. 👇show more

Flow matching produces smooth, deterministic trajectories. In contrast, the... sampling process of a diffusion model is chaotic, resembling the random motion of gas particles.show more

How can a visuomotor policy learn from internet videos?... We introduce Dreamitate, where a robot uses a fine-tuned video diffusion model to dream the future (top) and imitate the dream to accomplish a task (bottom). website: paper:show more

New ML gem on the hub: LDM3D by Intel.... This diffusion model generates image & depth from text prompts. Using a custom @gradio 6dof three.js component you can generate immersive 360-degree views from prompts demo: model:show more

animation test I did months ago for Kaycem 's... project, Ground Zeroes! The animatic just came out, so please check it out through them when you guys can!show more

[LIVE DEMO] Check out our latest WebGL demo, exploring... a fluid cellular particle effect. Full write up and details of what's going on under the hood coming soon 🔍 Live → #webgl #realtime #threejsshow more

YAY!!! consume me demo is ✨OUT NOW💖! you can... finally see some of the art i've been working on for the past two-ish years…🥴 it's been so much fun and everyone has worked so hard! i promise it's such a special game so please check out the demo and wishlist on steam!!show more

Chop the gradients ✂️! We found that truncating decoder... gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.show more

ANNOUNCEMENT Now anyone can use Stable Diffusion XL, our... latest text-to-image generative AI model, on our Clipdrop platform, for free! Test the limits of your creativity. #SDXL Try it here →show more

We have released the code and weights for our... #CVPR2023 paper "Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model"! code: abs: project: The demo is below:show more

P2P Model (sd15_ip2p) "Make he ironman" The ironman suit... is complex with a lot of lines and details but even so the model kept the coherence as best as it could without changing the background (only the color). Stable Diffusion #midjourney #ai #controlnet #animation #aiartshow more

Completed a 20 year dream of making my own... software. All using tech I couldn’t imagine would be possible - Claude Anthropic Perplexity I’m so proud of this and I hope you check it out. Epilogue is official live in the App Storeshow more

I’ve figured out an intuitive way to make consistent... characters and turn them into UGC This is Luna and I built her in Gluely. Check the demo to see how easy it is once you have an established charactershow more

Most recent diffusion language model research (that I’ve seen)... seems to be using masking as the noising process. It looks like, however, most closed-source models (Google Gemini Diffusion and possibly Inception Labs’ Mercury) use a different noising process, where instead of masking tokens, they replace them with different tokens (either with a random token or a semantically similar token). I wondered how they were getting such high throughput with the latter noising process, since I believed that optimizing inference with KVCache approximation would be more difficult (for various reasons). I visualized this noising process with tiny-diffusion and compared it to normal unmasking, and was very surprised to see how fast the generation “settles” into a reasonable output, and then only slightly refines afterwards, requiring much fewer steps in total. Unmasking (where tokens are never remasked, the typical implementation) is inherently limited in generation speed by the fact that an increase in tokens decoded per step leads to more errors due to the mismatch between individual and marginal token probability distributions we sample from. The token replacement noising process seems to have a much different set of characteristics. Because we sample each token per step, every token makes “progress” towards the final output each iteration (in addition to *potentially* giving other tokens more information in future steps). Generally, masking has outperformed other noising processes, which is probably why most research focused on it (using smaller models). But the paper referred to in the retweet shows that random replacement as a noising process may scale better as model size increases. Big labs might have noticed these results much earlier (due to having drastically more training resources and being able to test larger models), which may explain the discrepancy in the choice of noising process. I’m gonna test this with larger models, since tiny-diffusion only has 10M parameters.show more