Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

I finally released my new video on YouTube about Diffusion Models / Score-Based Generative Models. Literally planned this for a year and put so much work in. I think this approach to diffusion models is so intuitive and highly recommend giving that a go! Video is 38min long, so... show more

dome | Outlier

3,034 subscribers

54,611 Aufrufe • vor 2 Jahren •via X (Twitter)

Wissenschaft & Technologie Musik Bildung

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von 1LittleCoder💻

1LittleCoder💻vor 2 Jahren

That's a crazy timeline. Respect!

Profilbild von dome | Outlier

dome | Outliervor 2 Jahren

Thank you haha

Profilbild von Robin Rombach

Robin Rombachvor 2 Jahren

Nice, richtig gut 😍

Profilbild von dome | Outlier

dome | Outliervor 2 Jahren

Nächstes video dann über Flow Matching. Ich werd auf dich zurückkommen mit paar Fragen :c

Profilbild von Ryan Tabrizi

Ryan Tabrizivor 2 Jahren

Learned a lot following your derivations. Thanks!

Profilbild von dome | Outlier

dome | Outliervor 2 Jahren

So awesome to hear. One small addition, the solution to this is not just Sliced Score Matching. It‘s what Pascal Vincent came up with when he combined Score Matching & Denoising Autoencoders -> Denoising Score Matching So basically two solutions to this problem, and mostly DSM is used :) Super happy the video was helpful

Profilbild von Ariel N. Lee

Ariel N. Leevor 2 Jahren

This is awesome

Profilbild von Ashutosh Narang

Ashutosh Narangvor 2 Jahren

@jeremyphoward Watching it today, thank you

Profilbild von The Variational Book

The Variational Bookvor 2 Jahren

@dome_271 looks great, what did you use to make the video? Seems like would be right up your alley

Profilbild von dome | Outlier

dome | Outliervor 2 Jahren

Manim and Premiere Pro :c

Ähnliche Videos

Diffusion clicked for me when I read about score-based models, a line of work pioneered by Stefano Ermon (et al.) at Stanford. So it was a full-circle moment to collab with him and Inception on a video about training & sampling techniques for making diffusion LLMs faster.

Diffusion clicked for me when I read about score-based models, a line of work pioneered by Stefano Ermon (et al.) at Stanford. So it was a full-circle moment to collab with him and Inception on a video about training & sampling techniques for making diffusion LLMs faster.

Julia Turc

27,359 Aufrufe • vor 9 Monaten

Dreamix: Video Diffusion Models are General Video Editors abs: project page: present diffusion-based method that is able to perform text-based motion and appearance editing of general videos

Dreamix: Video Diffusion Models are General Video Editors abs: project page: present diffusion-based method that is able to perform text-based motion and appearance editing of general videos

AK

398,258 Aufrufe • vor 3 Jahren

Announcing Diffusion Forcing Transformer (DFoT), our new video diffusion algorithm that generates ultra-long videos of 800+ frames. DFoT enables History Guidance, a simple add-on to any existing video diffusion models for a quality boost. Website: (1/7)

Announcing Diffusion Forcing Transformer (DFoT), our new video diffusion algorithm that generates ultra-long videos of 800+ frames. DFoT enables History Guidance, a simple add-on to any existing video diffusion models for a quality boost. Website: (1/7)

Boyuan Chen

176,540 Aufrufe • vor 1 Jahr

This is the first time in a long while that I’ve let myself think about this again and rewatch this video since the first time. It hurt so much I could barely handle it…. Thank you so much Seungkwan and 17, i love you guys so much

This is the first time in a long while that I’ve let myself think about this again and rewatch this video since the first time. It hurt so much I could barely handle it…. Thank you so much Seungkwan and 17, i love you guys so much

🐻🐻‍❄️

99,886 Aufrufe • vor 9 Monaten

In 2022, we made a breakthrough in 2D jigsaw puzzles by diffusion models (Puzzle Fusion). Now, we make a breakthrough in 3D shape assembly with an auto-agglomerative approach based on diffusion models. Find the code, models, data, and paper at

In 2022, we made a breakthrough in 2D jigsaw puzzles by diffusion models (Puzzle Fusion). Now, we make a breakthrough in 3D shape assembly with an auto-agglomerative approach based on diffusion models. Find the code, models, data, and paper at

Yasutaka Furukawa

11,314 Aufrufe • vor 2 Jahren

MiniMax (official) H3 model is some of the most fun iv had in a long time in diffusion. Open source Has been stale for video for so long. This allows people to make some seriously impressive stuff. This is a single render, 1088x1920. I built a custom prompt scheduling system that allows me to schedule conditioning (prompts) to cut in and out as i please. So naturally i made it beat synced to t he song. There is zero editing done to this video

MiniMax (official) H3 model is some of the most fun iv had in a long time in diffusion. Open source Has been stale for video for so long. This allows people to make some seriously impressive stuff. This is a single render, 1088x1920. I built a custom prompt scheduling system that allows me to schedule conditioning (prompts) to cut in and out as i please. So naturally i made it beat synced to t he song. There is zero editing done to this video

Machine Delusions

33,004 Aufrufe • vor 3 Monaten

"Closed models are cheaper. The reason you need open models is that they give you control". - Nvidia CEO Jensen Huang "One of the things that people misunderstand about open models is this: yes, you can host them yourself, and you can build your own computer. But most people use computers in the cloud. Frankly, I think closed models are cheaper. If you do not build everything yourself, you avoid the significant cost and expertise required to train, fine-tune, maintain, guardrail, evaluate, and keep the model safe. You would also need to build or obtain the computers required to host it. So, there is nothing cheap about doing all of that yourself. The reason you need open models is that they give you control. You can adapt them for your own highly specialized use cases. I think there is a lot of misunderstanding about closed models versus open models. We felt it was important for people to understand that there is a place for both." ---- From "Bloomberg Podcasts" YouTube channel, (full video link in comment)

"Closed models are cheaper. The reason you need open models is that they give you control". - Nvidia CEO Jensen Huang "One of the things that people misunderstand about open models is this: yes, you can host them yourself, and you can build your own computer. But most people use computers in the cloud. Frankly, I think closed models are cheaper. If you do not build everything yourself, you avoid the significant cost and expertise required to train, fine-tune, maintain, guardrail, evaluate, and keep the model safe. You would also need to build or obtain the computers required to host it. So, there is nothing cheap about doing all of that yourself. The reason you need open models is that they give you control. You can adapt them for your own highly specialized use cases. I think there is a lot of misunderstanding about closed models versus open models. We felt it was important for people to understand that there is a place for both." ---- From "Bloomberg Podcasts" YouTube channel, (full video link in comment)

Rohan Paul

32,317 Aufrufe • vor 2 Monaten

Energy-Based Models can now scale better than Diffusion models. Introducing TEM: Transformer Energy-based Models TLDR: • TEM outscales Diffusion without guidance • First scalable contrastive Energy-Based Models • Better generalization, test-time scaling, and OOD detection than Diffusion 🧵

Energy-Based Models can now scale better than Diffusion models. Introducing TEM: Transformer Energy-based Models TLDR: • TEM outscales Diffusion without guidance • First scalable contrastive Energy-Based Models • Better generalization, test-time scaling, and OOD detection than Diffusion 🧵

Alexi Gladstone

101,398 Aufrufe • vor 11 Tagen

Nvidia presents Align Your Steps Optimizing Sampling Schedules in Diffusion Models Diffusion models (DMs) have established themselves as the state-of-the-art generative modeling approach in the visual domain and beyond. A crucial drawback of DMs is their slow sampling speed,

Nvidia presents Align Your Steps Optimizing Sampling Schedules in Diffusion Models Diffusion models (DMs) have established themselves as the state-of-the-art generative modeling approach in the visual domain and beyond. A crucial drawback of DMs is their slow sampling speed,

AK

32,888 Aufrufe • vor 2 Jahren

working towards action-conditioned video diffusion models for now this is just cs2 gameplay, and i parse the keypresses from the .dem game file next steps will be to work on a scalable data loader then coding the model

working towards action-conditioned video diffusion models for now this is just cs2 gameplay, and i parse the keypresses from the .dem game file next steps will be to work on a scalable data loader then coding the model

Arnie Ramesh

31,148 Aufrufe • vor 8 Monaten

Synthesizing worlds with video diffusion models is often inconsistent — moving the camera back and forth leads to different scenes. We propose 🌐𝗪𝗼𝗿𝗹𝗱𝗠𝗲𝗺, a memory-based approach that ensures consistent world simulation without relying on explicit 3D reconstruction.

Synthesizing worlds with video diffusion models is often inconsistent — moving the camera back and forth leads to different scenes. We propose 🌐𝗪𝗼𝗿𝗹𝗱𝗠𝗲𝗺, a memory-based approach that ensures consistent world simulation without relying on explicit 3D reconstruction.

Xingang Pan

19,413 Aufrufe • vor 1 Jahr

Excited to share our interview with Sander Dieleman! This is Part 3 in our History of Diffusion series. We talk about diffusion as spectral autoregression, diffusion language models, flow matching, and much more. Enjoy!

Excited to share our interview with Sander Dieleman! This is Part 3 in our History of Diffusion series. We talk about diffusion as spectral autoregression, diffusion language models, flow matching, and much more. Enjoy!

Slater Stich

28,721 Aufrufe • vor 1 Jahr

Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models paper page: Traditional 3D content creation tools empower users to bring their imagination to life by giving them direct control over a scene's geometry, appearance, motion, and camera path. Creating computer-generated videos, however, is a tedious manual process, which can be automated by emerging text-to-video diffusion models. Despite great promise, video diffusion models are difficult to control, hindering a user to apply their own creativity rather than amplifying it. To address this challenge, we present a novel approach that combines the controllability of dynamic 3D meshes with the expressivity and editability of emerging diffusion models. For this purpose, our approach takes an animated, low-fidelity rendered mesh as input and injects the ground truth correspondence information obtained from the dynamic mesh into various stages of a pre-trained text-to-image generation model to output high-quality and temporally consistent frames. We demonstrate our approach on various examples where motion can be obtained by animating rigged assets or changing the camera path.

Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models paper page: Traditional 3D content creation tools empower users to bring their imagination to life by giving them direct control over a scene's geometry, appearance, motion, and camera path. Creating computer-generated videos, however, is a tedious manual process, which can be automated by emerging text-to-video diffusion models. Despite great promise, video diffusion models are difficult to control, hindering a user to apply their own creativity rather than amplifying it. To address this challenge, we present a novel approach that combines the controllability of dynamic 3D meshes with the expressivity and editability of emerging diffusion models. For this purpose, our approach takes an animated, low-fidelity rendered mesh as input and injects the ground truth correspondence information obtained from the dynamic mesh into various stages of a pre-trained text-to-image generation model to output high-quality and temporally consistent frames. We demonstrate our approach on various examples where motion can be obtained by animating rigged assets or changing the camera path.

AK

117,091 Aufrufe • vor 2 Jahren

i asked astra 6 to create a video about "time" and it used p5.brush to make an entire video from big bang to evolution to civilization to astra creating the video using code (why does ai love reference) every frame is rendered in 4k using javascript, meaning, everything is editable in layers, keyframes and splines! coding models > diffusion models

i asked astra 6 to create a video about "time" and it used p5.brush to make an entire video from big bang to evolution to civilization to astra creating the video using code (why does ai love reference) every frame is rendered in 4k using javascript, meaning, everything is editable in layers, keyframes and splines! coding models > diffusion models

poobesh

18,860 Aufrufe • vor 1 Monat

I finally found the time and courage to spend a few minutes on what I really think. After the announcement of the end of #Destiny2 I laid bare what I feel. The video is long, so you’ll find the full version on my #YouTube channel. (The video is in Italian) This is for you, Guardians. Thank you for everything, and thank you, Destiny. A Light that tears through the darkness. I love you. Full Video: #Destiny2Creator #Twitch #TwitchItalia

I finally found the time and courage to spend a few minutes on what I really think. After the announcement of the end of #Destiny2 I laid bare what I feel. The video is long, so you’ll find the full version on my #YouTube channel. (The video is in Italian) This is for you, Guardians. Thank you for everything, and thank you, Destiny. A Light that tears through the darkness. I love you. Full Video: #Destiny2Creator #Twitch #TwitchItalia

IFreddo

10,581 Aufrufe • vor 5 Monaten

A list of all my profiles and the instructions on messaging me! This is very important so I highly recommend you watch and read through the whole video!

Sensitive content

A list of all my profiles and the instructions on messaging me! This is very important so I highly recommend you watch and read through the whole video!

KillerGirlSoles

34,280 Aufrufe • vor 2 Monaten

AI models pushing the frontier are a growing challenge for cybersecurity. A few weeks ago, I asked Demis what's underhyped in AI right now and on his mind: "I'm very excited about this new agentic era and you can see us leaning into that" "But of course we've also gotta think about the security side of that, too" "You're seeing it a little bit with the cyber worries about some of the models, and I think that's just the beginning of some of the issues that we need to make sure we guard against" "So I think that's playing a little bit on my mind. Maybe this is the time now to try and push some standards and maybe international cooperation"

AI models pushing the frontier are a growing challenge for cybersecurity. A few weeks ago, I asked Demis what's underhyped in AI right now and on his mind: "I'm very excited about this new agentic era and you can see us leaning into that" "But of course we've also gotta think about the security side of that, too" "You're seeing it a little bit with the cyber worries about some of the models, and I think that's just the beginning of some of the issues that we need to make sure we guard against" "So I think that's playing a little bit on my mind. Maybe this is the time now to try and push some standards and maybe international cooperation"

Rowan Cheung

36,216 Aufrufe • vor 3 Monaten

Glad that our work “Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling”, led by Han Qi, has been accepted to IEEE Robotics and Automation Letters! 🎉 We propose Generative Predictive Control (GPC): sample action proposals from a pretrained diffusion policy (“look back”), roll them out with a diffusion-based action-conditioned video world model (“look forward”), then rank or optimize the actions using either a learned reward model or VLM preferences. Conceptually, this is trajectory optimization / MPC with hybrid sampling + gradient optimization, interpreted through modern diffusion priors and video world models. Interestingly, we first posted the paper on arXiv in Feb 2025, when action-conditioned video world models for planning were still rare—now this direction is rapidly gaining traction. Still many open questions, e.g., • how to avoid local minima in planning • what representations work best for world models • how to balance physics priors vs. data-driven learning Paper:

Glad that our work “Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling”, led by Han Qi, has been accepted to IEEE Robotics and Automation Letters! 🎉 We propose Generative Predictive Control (GPC): sample action proposals from a pretrained diffusion policy (“look back”), roll them out with a diffusion-based action-conditioned video world model (“look forward”), then rank or optimize the actions using either a learned reward model or VLM preferences. Conceptually, this is trajectory optimization / MPC with hybrid sampling + gradient optimization, interpreted through modern diffusion priors and video world models. Interestingly, we first posted the paper on arXiv in Feb 2025, when action-conditioned video world models for planning were still rare—now this direction is rapidly gaining traction. Still many open questions, e.g., • how to avoid local minima in planning • what representations work best for world models • how to balance physics priors vs. data-driven learning Paper:

Heng Yang

19,293 Aufrufe • vor 8 Monaten

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

AK

143,557 Aufrufe • vor 3 Jahren

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,681 Aufrufe • vor 3 Jahren