Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Google presents LightLab Controlling Light Sources in Images with Diffusion Models

AK

502,106 subscribers

122,958 views • 1 year ago •via X (Twitter)

Science & Technology News & Politics Education

Anya Rossi• Live Now

Private livecam show

5 Comments

AK1 year ago

discuss with author:

revolver ocelot1 year ago

Kinda similar to relight from SD

Giulio Aprin1 year ago

@WilliamLamkin Wow

J1 year ago

cool

C12s1 year ago

Game changer for image editing.

Related Videos

Google published LightLab! a new method that can change the intensity and color of light sources in images and adjust ambient lighting

Google published LightLab! a new method that can change the intensity and color of light sources in images and adjust ambient lighting

Dreaming Tulpa 🥓👑

35,279 views • 1 year ago

Google presents CAT4D Create Anything in 4D with Multi-View Video Diffusion Models

Google presents CAT4D Create Anything in 4D with Multi-View Video Diffusion Models

AK

61,949 views • 1 year ago

Nvidia presents Articulated Kinematics Distillation from Video Diffusion Models

Nvidia presents Articulated Kinematics Distillation from Video Diffusion Models

AK

39,189 views • 1 year ago

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,657 views • 2 years ago

Nvidia presents Align Your Steps Optimizing Sampling Schedules in Diffusion Models Diffusion models (DMs) have established themselves as the state-of-the-art generative modeling approach in the visual domain and beyond. A crucial drawback of DMs is their slow sampling speed,

Nvidia presents Align Your Steps Optimizing Sampling Schedules in Diffusion Models Diffusion models (DMs) have established themselves as the state-of-the-art generative modeling approach in the visual domain and beyond. A crucial drawback of DMs is their slow sampling speed,

AK

32,888 views • 2 years ago

Oh wow! "[SIGGRAPH '24] DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models" Paper: Project: Code (MIT) :

Oh wow! "[SIGGRAPH '24] DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models" Paper: Project: Code (MIT) :

MrNeRF

59,480 views • 2 years ago

Meta presents Adaptive Caching for Faster Video Generation with Diffusion Transformers

Meta presents Adaptive Caching for Faster Video Generation with Diffusion Transformers

AK

53,119 views • 1 year ago

Diffusion models generate high-quality images but require hundreds of forward passes. MIT CSAIL and Adobe Research introduce Distribution Matching Distillation (DMD), a distillation approach that converts costly multi-step diffusion models into fast one-step generators. A thread 🧵

Diffusion models generate high-quality images but require hundreds of forward passes. MIT CSAIL and Adobe Research introduce Distribution Matching Distillation (DMD), a distillation approach that converts costly multi-step diffusion models into fast one-step generators. A thread 🧵

MIT CSAIL

34,347 views • 2 years ago

Block Diffusion Interpolating Between Autoregressive and Diffusion Language Models

Block Diffusion Interpolating Between Autoregressive and Diffusion Language Models

AK

160,553 views • 1 year ago

Google isn’t betting on a single AI architecture. Sundar Pichai, CEO of Google: “We’re going to push the diffusion paradigm as hard as possible.” “All of today’s mainline Gemini models are autoregressive. Diffusion is a different paradigm.” “For the same capability, diffusion can be much faster.” “It’s behind the mainline models today, but there will be areas where it’s the right tool.” “We’re pushing multiple directions in parallel, and bringing them together where it makes sense.”

Google isn’t betting on a single AI architecture. Sundar Pichai, CEO of Google: “We’re going to push the diffusion paradigm as hard as possible.” “All of today’s mainline Gemini models are autoregressive. Diffusion is a different paradigm.” “For the same capability, diffusion can be much faster.” “It’s behind the mainline models today, but there will be areas where it’s the right tool.” “We’re pushing multiple directions in parallel, and bringing them together where it makes sense.”

Forward Future

152,625 views • 5 months ago

Excited to share our interview with Sander Dieleman! This is Part 3 in our History of Diffusion series. We talk about diffusion as spectral autoregression, diffusion language models, flow matching, and much more. Enjoy!

Excited to share our interview with Sander Dieleman! This is Part 3 in our History of Diffusion series. We talk about diffusion as spectral autoregression, diffusion language models, flow matching, and much more. Enjoy!

Slater Stich

28,713 views • 1 year ago

Guidance on top of diffusion models can now be used to drag and manipulate images, create pose-conditioned images, and so much more! Check out Readout Guidance: Work w/ trevordarrell, Oliver Wang, Dan Goldman, Aleksander Holynski. More in thread 🧵.

Guidance on top of diffusion models can now be used to drag and manipulate images, create pose-conditioned images, and so much more! Check out Readout Guidance: Work w/ trevordarrell, Oliver Wang, Dan Goldman, Aleksander Holynski. More in thread 🧵.

Grace Luo

42,698 views • 2 years ago

Today, we’re introducing Video Overviews in NotebookLM — a visual alternative to Audio Overviews. Dive deeper with short, engaging slide summaries using images, diagrams, quotes and data from your sources, narrated by your AI host.

Today, we’re introducing Video Overviews in NotebookLM — a visual alternative to Audio Overviews. Dive deeper with short, engaging slide summaries using images, diagrams, quotes and data from your sources, narrated by your AI host.

Google

656,241 views • 10 months ago

DiffSplat Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation DiffSplat is a generative framework to synthesize 3D Gaussian Splats from text prompts & single-view images in ⚡️ 1~2 seconds. It is fine-tuned directly from a pretrained text-to-image diffusion model.

DiffSplat Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation DiffSplat is a generative framework to synthesize 3D Gaussian Splats from text prompts & single-view images in ⚡️ 1~2 seconds. It is fine-tuned directly from a pretrained text-to-image diffusion model.

AK

38,416 views • 1 year ago

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:

AK

304,708 views • 3 years ago

Today, we’re launching our newest gen AI experiment in @labsdotgoogle: Whisk. Instead of generating images with long, detailed text prompts, Whisk lets you prompt with images. Simply drag in images, and start creating.

Today, we’re launching our newest gen AI experiment in @labsdotgoogle: Whisk. Instead of generating images with long, detailed text prompts, Whisk lets you prompt with images. Simply drag in images, and start creating.

Google

259,790 views • 1 year ago

PAIR-Diffusion: Object-Level Image Editing with Structure-and-Appearance Paired Diffusion Models Gradio demo is out on Hugging Face Spaces demo:

PAIR-Diffusion: Object-Level Image Editing with Structure-and-Appearance Paired Diffusion Models Gradio demo is out on Hugging Face Spaces demo:

AK

87,679 views • 3 years ago

Nvidia presents ConsiStory Training-Free Consistent Text-to-Image Generation paper page: enable Stable Diffusion XL (SDXL) to generate consistent subjects across a series of images, without additional training.

Nvidia presents ConsiStory Training-Free Consistent Text-to-Image Generation paper page: enable Stable Diffusion XL (SDXL) to generate consistent subjects across a series of images, without additional training.

AK

161,673 views • 2 years ago

🤯 Gemini 3.1 Pro Google DeepMind generates parametric 3D models straight from images! 🚀Paired with Rodin Gen-2 Edit's Refine, #Gemini effortlessly delivers million-face detailed #3D models. 🔥3D Generation workflows are evolving! See it in action 👇 #CG #Hyper3D

🤯 Gemini 3.1 Pro Google DeepMind generates parametric 3D models straight from images! 🚀Paired with Rodin Gen-2 Edit's Refine, #Gemini effortlessly delivers million-face detailed #3D models. 🔥3D Generation workflows are evolving! See it in action 👇 #CG #Hyper3D

Hyper3D by Deemos

77,496 views • 3 months ago