Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Diffusion is the foundational ML framework behind state-of-the-art AI image and video generation, including Sora, Midjourney and Google Veo. In this episode of Decoded, Ankit Gupta sits down with Francois Chaubard to discuss how diffusion works, walk through a code sample, and explain why everyone training models should understand...

ycombinator's profile picture

Y Combinator

1,539,544 subscribers

74,604 Aufrufe • vor 4 Monaten •via X (Twitter)

BildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

This video explains how diffusion models are overtaking Large Language Models for generation tasks like: 1. Code Generation 2. Image Generation 3. Video Generation 00:00 Agenda 00:20 How are they different from LLMs? 05:09 Internal Mechanism 10:09 How are vectors generated? 12:08 Conclusion 13:02 Opinion Piece AI Engineering Course: #Diffusion #AI #LLMs
15:00

This video explains how diffusion models are overtaking Large Language Models for generation tasks like: 1. Code Generation 2. Image Generation 3. Video Generation 00:00 Agenda 00:20 How are they different from LLMs? 05:09 Internal Mechanism 10:09 How are vectors generated? 12:08 Conclusion 13:02 Opinion Piece AI Engineering Course: #Diffusion #AI #LLMs

Gaurav Sen

27,910 Aufrufe • vor 7 Monaten

This is getting interesting. In this iteration, the magnifying glass is tracked in real-time, feeding only what is seen through it to the diffusion process, revealing the invisible within a piece of clay. Real-time diffusion with Stream Diffusion + SDXL Turbo. #StableDiffusion #SDXLTurbo #StreamDiffusion #StabilityAI #Realtime
1:16

This is getting interesting. In this iteration, the magnifying glass is tracked in real-time, feeding only what is seen through it to the diffusion process, revealing the invisible within a piece of clay. Real-time diffusion with Stream Diffusion + SDXL Turbo. #StableDiffusion #SDXLTurbo #StreamDiffusion #StabilityAI #Realtime

Hugues Bruyère

26,432 Aufrufe • vor 2 Jahren

I'm thrilled to announce the launch of ⚡️Flash Diffusion from Jasper! Earlier this year, with our acquisition of Clipdrop, we launched the Jasper AI Research Lab in Paris. Today, we are excited to release our first piece of groundbreaking research: the open-source distillation method, "Flash Diffusion". Flash Diffusion accelerates inference by 500%, reduces computing costs, and produces higher-quality image outputs. Dive into the details and discover how Flash Diffusion is set to revolutionize the field of AI and image synthesis. Read all about it here: Try a demo on Hugging Face:
0:28

I'm thrilled to announce the launch of ⚡️Flash Diffusion from Jasper! Earlier this year, with our acquisition of Clipdrop, we launched the Jasper AI Research Lab in Paris. Today, we are excited to release our first piece of groundbreaking research: the open-source distillation method, "Flash Diffusion". Flash Diffusion accelerates inference by 500%, reduces computing costs, and produces higher-quality image outputs. Dive into the details and discover how Flash Diffusion is set to revolutionize the field of AI and image synthesis. Read all about it here: Try a demo on Hugging Face:

Timothy Young

10,062 Aufrufe • vor 2 Jahren

Google isn’t betting on a single AI architecture. Sundar Pichai, CEO of Google: “We’re going to push the diffusion paradigm as hard as possible.” “All of today’s mainline Gemini models are autoregressive. Diffusion is a different paradigm.” “For the same capability, diffusion can be much faster.” “It’s behind the mainline models today, but there will be areas where it’s the right tool.” “We’re pushing multiple directions in parallel, and bringing them together where it makes sense.”
1:02

Google isn’t betting on a single AI architecture. Sundar Pichai, CEO of Google: “We’re going to push the diffusion paradigm as hard as possible.” “All of today’s mainline Gemini models are autoregressive. Diffusion is a different paradigm.” “For the same capability, diffusion can be much faster.” “It’s behind the mainline models today, but there will be areas where it’s the right tool.” “We’re pushing multiple directions in parallel, and bringing them together where it makes sense.”

Forward Future

152,625 Aufrufe • vor 5 Monaten

Nvidia presents Align Your Steps Optimizing Sampling Schedules in Diffusion Models Diffusion models (DMs) have established themselves as the state-of-the-art generative modeling approach in the visual domain and beyond. A crucial drawback of DMs is their slow sampling speed,
0:34

Nvidia presents Align Your Steps Optimizing Sampling Schedules in Diffusion Models Diffusion models (DMs) have established themselves as the state-of-the-art generative modeling approach in the visual domain and beyond. A crucial drawback of DMs is their slow sampling speed,

AK

32,888 Aufrufe • vor 2 Jahren

Wow, diffusion models (used in AI image generation) are also game engines - a type of world simulation. By predicting the next frame of the classic shooter DOOM, you get a playable game at 20 fps without any underlying real game engine. This video is from the diffusion model.
2:30

Wow, diffusion models (used in AI image generation) are also game engines - a type of world simulation. By predicting the next frame of the classic shooter DOOM, you get a playable game at 20 fps without any underlying real game engine. This video is from the diffusion model.

Ethan Mollick

1,768,587 Aufrufe • vor 1 Jahr

Excited to share our interview with Sander Dieleman! This is Part 3 in our History of Diffusion series. We talk about diffusion as spectral autoregression, diffusion language models, flow matching, and much more. Enjoy!
45:04

Excited to share our interview with Sander Dieleman! This is Part 3 in our History of Diffusion series. We talk about diffusion as spectral autoregression, diffusion language models, flow matching, and much more. Enjoy!

Slater Stich

28,693 Aufrufe • vor 1 Jahr

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models turn the publicly available, state-of-the-art text-to-image LDM Stable Diffusion into an efficient and expressive text-to-video model with resolution up to 1280 x 2048 abs: project page:
1:38

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models turn the publicly available, state-of-the-art text-to-image LDM Stable Diffusion into an efficient and expressive text-to-video model with resolution up to 1280 x 2048 abs: project page:

AK

718,682 Aufrufe • vor 3 Jahren

*New Lecture* Stanford Frontier Systems, Session 4 (Full Video) Frontier Visual Intelligence Systems with Andreas Blattmann from BlackForestLabsAI - Unofficial 00:00 Bella Napoli 00:56 Frontier Flywheel Recap 04:29 Intro to Andi 06:26 From GANs to Latent Diffusion 08:31 Stable Diffusion Breakthrough 15:20 Beyond Content Creation 18:28 Starting Black Forest Labs 21:07 Flux One Training Pipeline 24:15 Character Consistency Leap 27:02 Leadership Under Pressure 30:40 What Comes Next 32:06 Training to Action 41:07 Self Flow Intuition 48:35 Labeling and Distillation 57:09 3D vs Video Debate
1:01:09

*New Lecture* Stanford Frontier Systems, Session 4 (Full Video) Frontier Visual Intelligence Systems with Andreas Blattmann from BlackForestLabsAI - Unofficial 00:00 Bella Napoli 00:56 Frontier Flywheel Recap 04:29 Intro to Andi 06:26 From GANs to Latent Diffusion 08:31 Stable Diffusion Breakthrough 15:20 Beyond Content Creation 18:28 Starting Black Forest Labs 21:07 Flux One Training Pipeline 24:15 Character Consistency Leap 27:02 Leadership Under Pressure 30:40 What Comes Next 32:06 Training to Action 41:07 Self Flow Intuition 48:35 Labeling and Distillation 57:09 3D vs Video Debate

Anjney Midha

73,723 Aufrufe • vor 1 Monat

PAIR-Diffusion: Object-Level Image Editing with Structure-and-Appearance Paired Diffusion Models Gradio demo is out on Hugging Face Spaces demo:
1:05

PAIR-Diffusion: Object-Level Image Editing with Structure-and-Appearance Paired Diffusion Models Gradio demo is out on Hugging Face Spaces demo:

AK

87,679 Aufrufe • vor 3 Jahren

My conversation with OpenAI co-founder Greg Brockman This is the most detailed first-person account of the 72 hours after Sam Altman was fired. We also go deep on what comes next: the global race to AGI, why ChatGPT stopped showing reasoning, how much of OpenAI's own code is now written by AI ("it's hard to know what percent is not"), and the untold story of how OpenAI actually started in 2015. 00:00:00 Introduction 00:00:49 Meeting Sam Altman and Starting OpenAI 00:02:40 Building the Founding Team 00:04:25 DeepMind's Lead Over OpenAI 00:04:54 Changing OpenAI to a For-Profit Model 00:06:05 Breakthrough Moments at OpenAI 00:08:22 What Dota 2 Meant for OpenAI 00:10:04 Reasoning Versus Prediction 00:11:59 Tensions Grow at OpenAI 00:15:44 Sam Altman's Firing 00:17:49 Greg Quits OpenAI 00:19:56 Sam Explores Deal with Microsoft's Satya 00:20:28 Petition for Altman's Return 00:23:43 Ilya Sutskever Leaves OpenAI 00:24:59 Lessons Learned after Sam Ousting 00:28:22 The Thing Ilya Said that Greg Can't Forget 00:32:22 Is AI Going Parabolic? 00:33:24 How Much of OpenAI's Code is Written by AI? 00:36:21 Do AI Chatbots Tell Us What We Want to Hear? 00:38:06 The Global AI Race to Reach AGI 00:38:40 What Happens if US Doesn't Reach AGI First? 00:39:49 Are Countries Stealing AI Advancements? 00:40:38 Why ChatGPT No Longer Shows Reasoning 00:41:47 The Finite Constraints of Compute 00:43:38 On Investing Early in Data Centers 00:46:31 The Future of Data Center Specialization 00:47:52 How to Decide Whose Queries to Serve 00:49:08 OpenAI on Consumer vs Enterprise Models 00:53:05 Data Centers in Space? 01:00:56 What Should AI Regulation Look Like? 01:04:33 The Future of AI-Powered Entrepreneurship 01:04:44 AI and Job Loss 01:07:15 The Skills Young People Should Invest In 01:11:30 What Does Success Look Like For You? Full episode on X below. Also find it on: • YouTube: • Spotify: • Apple:
1:11:41

My conversation with OpenAI co-founder Greg Brockman This is the most detailed first-person account of the 72 hours after Sam Altman was fired. We also go deep on what comes next: the global race to AGI, why ChatGPT stopped showing reasoning, how much of OpenAI's own code is now written by AI ("it's hard to know what percent is not"), and the untold story of how OpenAI actually started in 2015. 00:00:00 Introduction 00:00:49 Meeting Sam Altman and Starting OpenAI 00:02:40 Building the Founding Team 00:04:25 DeepMind's Lead Over OpenAI 00:04:54 Changing OpenAI to a For-Profit Model 00:06:05 Breakthrough Moments at OpenAI 00:08:22 What Dota 2 Meant for OpenAI 00:10:04 Reasoning Versus Prediction 00:11:59 Tensions Grow at OpenAI 00:15:44 Sam Altman's Firing 00:17:49 Greg Quits OpenAI 00:19:56 Sam Explores Deal with Microsoft's Satya 00:20:28 Petition for Altman's Return 00:23:43 Ilya Sutskever Leaves OpenAI 00:24:59 Lessons Learned after Sam Ousting 00:28:22 The Thing Ilya Said that Greg Can't Forget 00:32:22 Is AI Going Parabolic? 00:33:24 How Much of OpenAI's Code is Written by AI? 00:36:21 Do AI Chatbots Tell Us What We Want to Hear? 00:38:06 The Global AI Race to Reach AGI 00:38:40 What Happens if US Doesn't Reach AGI First? 00:39:49 Are Countries Stealing AI Advancements? 00:40:38 Why ChatGPT No Longer Shows Reasoning 00:41:47 The Finite Constraints of Compute 00:43:38 On Investing Early in Data Centers 00:46:31 The Future of Data Center Specialization 00:47:52 How to Decide Whose Queries to Serve 00:49:08 OpenAI on Consumer vs Enterprise Models 00:53:05 Data Centers in Space? 01:00:56 What Should AI Regulation Look Like? 01:04:33 The Future of AI-Powered Entrepreneurship 01:04:44 AI and Job Loss 01:07:15 The Skills Young People Should Invest In 01:11:30 What Does Success Look Like For You? Full episode on X below. Also find it on: • YouTube: • Spotify: • Apple:

Shane Parrish

445,472 Aufrufe • vor 1 Monat

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.
5:11

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,657 Aufrufe • vor 2 Jahren

🏃Motion Tokenizer Bridging Semantic and Kinematic Conditions🏃‍♀️ #MoTok is a diffusion-based tokenizer that unifies *perception-planning-control*, combining the strengths of continuous diffusion & discrete tokens - Project: - Code:
5:01

🏃Motion Tokenizer Bridging Semantic and Kinematic Conditions🏃‍♀️ #MoTok is a diffusion-based tokenizer that unifies *perception-planning-control*, combining the strengths of continuous diffusion & discrete tokens - Project: - Code:

Ziwei Liu

19,209 Aufrufe • vor 2 Monaten

Dreamix: Video Diffusion Models are General Video Editors abs: project page: present diffusion-based method that is able to perform text-based motion and appearance editing of general videos
1:53

Dreamix: Video Diffusion Models are General Video Editors abs: project page: present diffusion-based method that is able to perform text-based motion and appearance editing of general videos

AK

398,132 Aufrufe • vor 3 Jahren

Just found the dLLM library to create Diffusion Language Models It's still early but it's insanely fun to experiment with diffusion (training, inference, eval) dLLM has the potential of becoming the main library for diffusion LLMs
0:26

Just found the dLLM library to create Diffusion Language Models It's still early but it's insanely fun to experiment with diffusion (training, inference, eval) dLLM has the potential of becoming the main library for diffusion LLMs

Maxime Labonne

75,421 Aufrufe • vor 6 Monaten

DiffSplat Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation DiffSplat is a generative framework to synthesize 3D Gaussian Splats from text prompts & single-view images in ⚡️ 1~2 seconds. It is fine-tuned directly from a pretrained text-to-image diffusion model.
0:24

DiffSplat Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation DiffSplat is a generative framework to synthesize 3D Gaussian Splats from text prompts & single-view images in ⚡️ 1~2 seconds. It is fine-tuned directly from a pretrained text-to-image diffusion model.

AK

38,416 Aufrufe • vor 1 Jahr

Block Diffusion Interpolating Between Autoregressive and Diffusion Language Models
0:23

Block Diffusion Interpolating Between Autoregressive and Diffusion Language Models

AK

160,553 Aufrufe • vor 1 Jahr

YC F24's Diffusion Studio (Diffusion Studio) is Canva for video editing, enabling everyone to create professional videos using AI in minutes. Congrats on the launch, Matthias Alexander Ruiz and konstantinpaulus!
1:26

YC F24's Diffusion Studio (Diffusion Studio) is Canva for video editing, enabling everyone to create professional videos using AI in minutes. Congrats on the launch, Matthias Alexander Ruiz and konstantinpaulus!

Y Combinator

30,094 Aufrufe • vor 1 Jahr

V3D Video Diffusion Models are Effective 3D Generators Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency
0:21

V3D Video Diffusion Models are Effective 3D Generators Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency

AK

31,997 Aufrufe • vor 2 Jahren

Oh wow! "[SIGGRAPH '24] DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models" Paper: Project: Code (MIT) :
1:04

Oh wow! "[SIGGRAPH '24] DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models" Paper: Project: Code (MIT) :

MrNeRF

59,480 Aufrufe • vor 2 Jahren