Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

The Dream 7B (diffusion reasoning language model) is OUT! 🚨 I built a demo so you can test it out (and check the diffusion process live) 𖣯🔍

apolinario (poli)

15,759 subscribers

35,679 views • 1 year ago •via X (Twitter)

Arts Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

LLaDA (the first Large Language Diffusion Model) is *just* out 💥 and I've built a demo, try out now 👨‍💻 It's mesmerizing to watch the diffusion process 🌀, and it being a diffusion model gives you superpowers like "the 4th word has to be pineapple" 🦸 Demo and weights 👇

LLaDA (the first Large Language Diffusion Model) is just out 💥 and I've built a demo, try out now 👨‍💻 It's mesmerizing to watch the diffusion process 🌀, and it being a diffusion model gives you superpowers like "the 4th word has to be pineapple" 🦸 Demo and weights 👇

apolinario (poli)

82,599 views • 1 year ago

(1/2) Check out "𝐏𝐨𝐥𝐲𝐃𝐢𝐟𝐟: Generating 3D Polygonal Meshes with Diffusion Models"! Our model operates directly on the polygons of 3D meshes and generates novel shapes as output through an iterative diffusion process.

(1/2) Check out "𝐏𝐨𝐥𝐲𝐃𝐢𝐟𝐟: Generating 3D Polygonal Meshes with Diffusion Models"! Our model operates directly on the polygons of 3D meshes and generates novel shapes as output through an iterative diffusion process.

Matthias Niessner

57,947 views • 2 years ago

I made a tool called Diffusion Explorer that lets you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇

I made a tool called Diffusion Explorer that lets you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇

Alec Helbling

73,124 views • 1 year ago

Excited to introduce Diffusion Augmented Agents (DAAGs)✨. We give an agent control of a diffusion model, so it can create its own *synthetic experience*.🪄 The result is a lifelong agent that can learn new reward detectors and policies, much more efficiently. Here's how. 👇

Excited to introduce Diffusion Augmented Agents (DAAGs)✨. We give an agent control of a diffusion model, so it can create its own synthetic experience.🪄 The result is a lifelong agent that can learn new reward detectors and policies, much more efficiently. Here's how. 👇

Norman Di Palo

12,931 views • 2 years ago

Flow matching produces smooth, deterministic trajectories. In contrast, the sampling process of a diffusion model is chaotic, resembling the random motion of gas particles.

Flow matching produces smooth, deterministic trajectories. In contrast, the sampling process of a diffusion model is chaotic, resembling the random motion of gas particles.

Alec Helbling

436,518 views • 1 year ago

How can a visuomotor policy learn from internet videos? We introduce Dreamitate, where a robot uses a fine-tuned video diffusion model to dream the future (top) and imitate the dream to accomplish a task (bottom). website: paper:

How can a visuomotor policy learn from internet videos? We introduce Dreamitate, where a robot uses a fine-tuned video diffusion model to dream the future (top) and imitate the dream to accomplish a task (bottom). website: paper:

Ruoshi Liu

50,797 views • 2 years ago

New ML gem on the hub: LDM3D by Intel. This diffusion model generates image & depth from text prompts. Using a custom @gradio 6dof three.js component you can generate immersive 360-degree views from prompts demo: model:

New ML gem on the hub: LDM3D by Intel. This diffusion model generates image & depth from text prompts. Using a custom @gradio 6dof three.js component you can generate immersive 360-degree views from prompts demo: model:

Radamés Ajna

56,696 views • 3 years ago

[LIVE DEMO] Check out our latest WebGL demo, exploring a fluid cellular particle effect. Full write up and details of what's going on under the hood coming soon 🔍 Live → #webgl #realtime #threejs

[LIVE DEMO] Check out our latest WebGL demo, exploring a fluid cellular particle effect. Full write up and details of what's going on under the hood coming soon 🔍 Live → #webgl #realtime #threejs

Unseen Studio®

12,116 views • 1 year ago

animation test I did months ago for Kaycem 's project, Ground Zeroes! The animatic just came out, so please check it out through them when you guys can!

animation test I did months ago for Kaycem 's project, Ground Zeroes! The animatic just came out, so please check it out through them when you guys can!

Toniko Pantoja

18,726 views • 7 months ago

YAY!!! consume me demo is ✨OUT NOW💖! you can finally see some of the art i've been working on for the past two-ish years…🥴 it's been so much fun and everyone has worked so hard! i promise it's such a special game so please check out the demo and wishlist on steam!!

YAY!!! consume me demo is ✨OUT NOW💖! you can finally see some of the art i've been working on for the past two-ish years…🥴 it's been so much fun and everyone has worked so hard! i promise it's such a special game so please check out the demo and wishlist on steam!!

jie en

10,830 views • 1 year ago

Chop the gradients ✂️! We found that truncating decoder gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.

Chop the gradients ✂️! We found that truncating decoder gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.

Felix Heide

28,399 views • 4 months ago

ANNOUNCEMENT Now anyone can use Stable Diffusion XL, our latest text-to-image generative AI model, on our Clipdrop platform, for free! Test the limits of your creativity. #SDXL Try it here →

ANNOUNCEMENT Now anyone can use Stable Diffusion XL, our latest text-to-image generative AI model, on our Clipdrop platform, for free! Test the limits of your creativity. #SDXL Try it here →

Stability AI

348,520 views • 3 years ago

We have released the code and weights for our #CVPR2023 paper "Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model"! code: abs: project: The demo is below:

We have released the code and weights for our #CVPR2023 paper "Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model"! code: abs: project: The demo is below:

Artsiom Sanakoyeu

35,712 views • 3 years ago

P2P Model (sd15_ip2p) "Make he ironman" The ironman suit is complex with a lot of lines and details but even so the model kept the coherence as best as it could without changing the background (only the color). Stable Diffusion #midjourney #ai #controlnet #animation #aiart

P2P Model (sd15_ip2p) "Make he ironman" The ironman suit is complex with a lot of lines and details but even so the model kept the coherence as best as it could without changing the background (only the color). Stable Diffusion #midjourney #ai #controlnet #animation #aiart

Mrboofy

10,171 views • 3 years ago

Completed a 20 year dream of making my own software. All using tech I couldn’t imagine would be possible - Claude Anthropic Perplexity I’m so proud of this and I hope you check it out. Epilogue is official live in the App Store

Completed a 20 year dream of making my own software. All using tech I couldn’t imagine would be possible - Claude Anthropic Perplexity I’m so proud of this and I hope you check it out. Epilogue is official live in the App Store

Kris Puckett

14,026 views • 10 months ago

I’ve figured out an intuitive way to make consistent characters and turn them into UGC This is Luna and I built her in Gluely. Check the demo to see how easy it is once you have an established character

I’ve figured out an intuitive way to make consistent characters and turn them into UGC This is Luna and I built her in Gluely. Check the demo to see how easy it is once you have an established character

Michael Aubry

39,082 views • 7 months ago

Most recent diffusion language model research (that I’ve seen) seems to be using masking as the noising process. It looks like, however, most closed-source models (Google Gemini Diffusion and possibly Inception Labs’ Mercury) use a different noising process, where instead of masking tokens, they replace them with different tokens (either with a random token or a semantically similar token). I wondered how they were getting such high throughput with the latter noising process, since I believed that optimizing inference with KVCache approximation would be more difficult (for various reasons). I visualized this noising process with tiny-diffusion and compared it to normal unmasking, and was very surprised to see how fast the generation “settles” into a reasonable output, and then only slightly refines afterwards, requiring much fewer steps in total. Unmasking (where tokens are never remasked, the typical implementation) is inherently limited in generation speed by the fact that an increase in tokens decoded per step leads to more errors due to the mismatch between individual and marginal token probability distributions we sample from. The token replacement noising process seems to have a much different set of characteristics. Because we sample each token per step, every token makes “progress” towards the final output each iteration (in addition to *potentially* giving other tokens more information in future steps). Generally, masking has outperformed other noising processes, which is probably why most research focused on it (using smaller models). But the paper referred to in the retweet shows that random replacement as a noising process may scale better as model size increases. Big labs might have noticed these results much earlier (due to having drastically more training resources and being able to test larger models), which may explain the discrepancy in the choice of noising process. I’m gonna test this with larger models, since tiny-diffusion only has 10M parameters.

Most recent diffusion language model research (that I’ve seen) seems to be using masking as the noising process. It looks like, however, most closed-source models (Google Gemini Diffusion and possibly Inception Labs’ Mercury) use a different noising process, where instead of masking tokens, they replace them with different tokens (either with a random token or a semantically similar token). I wondered how they were getting such high throughput with the latter noising process, since I believed that optimizing inference with KVCache approximation would be more difficult (for various reasons). I visualized this noising process with tiny-diffusion and compared it to normal unmasking, and was very surprised to see how fast the generation “settles” into a reasonable output, and then only slightly refines afterwards, requiring much fewer steps in total. Unmasking (where tokens are never remasked, the typical implementation) is inherently limited in generation speed by the fact that an increase in tokens decoded per step leads to more errors due to the mismatch between individual and marginal token probability distributions we sample from. The token replacement noising process seems to have a much different set of characteristics. Because we sample each token per step, every token makes “progress” towards the final output each iteration (in addition to potentially giving other tokens more information in future steps). Generally, masking has outperformed other noising processes, which is probably why most research focused on it (using smaller models). But the paper referred to in the retweet shows that random replacement as a noising process may scale better as model size increases. Big labs might have noticed these results much earlier (due to having drastically more training resources and being able to test larger models), which may explain the discrepancy in the choice of noising process. I’m gonna test this with larger models, since tiny-diffusion only has 10M parameters.

nathan (in sf)

40,440 views • 7 months ago