Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

This is getting interesting. In this iteration, the magnifying glass is tracked in real-time, feeding only what is seen through it to the diffusion process, revealing the invisible within a piece of clay. Real-time diffusion with Stream Diffusion + SDXL Turbo. #StableDiffusion #SDXLTurbo #StreamDiffusion #StabilityAI #Realtime

Hugues Bruyère

7,299 subscribers

26,432 views • 2 years ago •via X (Twitter)

Science & Technology Arts #StableDiffusion

Anya Rossi• Live Now

Private livecam show

10 Comments

littlecakes2 years ago

amazing!

Hugues Bruyère2 years ago

Thank you!

alejandro cartagena2 years ago

❤️❤️❤️

Hugues Bruyère2 years ago

☺️😶‍🌫️

Brian Jordan2 years ago

@cumulo_autumn

Moritz Stefaner2 years ago

Lovely!

Hugues Bruyère2 years ago

Thanks!

Metaverse | Eric2 years ago

This is so great，Very interesting

humanboy2 years ago

💜

MouX2 years ago

This is the wow

Related Videos

In the past few weeks, I deep dived into an exploration revolving around the use of physical interfaces to feed and interact with a real-time img2img diffusion pipeline using Stream Diffusion and SDXL Turbo. What really captivated me is to use my hands, objects, art supplies, tools, and light to create images and scenes. 𝗣𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗠𝗲𝗱𝗶𝗮 + 𝗧𝗼𝗼𝗹𝘀 I experimented with clay, manipulating different types and colors along with a selection of prompts. I used a magnifying glass, tracked in real-time, to focus the diffusion process on specific areas. Combining these tools created a dynamic and inspiring experience. Using magic clay to layer shapes and colors as a base for revealing landscapes and hidden worlds, and the magnifying glass to focus and reveal these details, was particularly effective. 𝗣𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗟𝗶𝗴𝗵𝘁 I used light as my method of interaction with the img2img diffusion. This approach felt special right away. There was something magical about holding a physical light source and seeing it influence the generated visuals. I iterated on this technique with themes like Rococo architecture, flowers, Brutalist architecture, hidden worlds, and origami landscapes. 𝗜𝗻𝗸 + 𝗛𝘆𝗯𝗿𝗶𝗱 𝗙𝗼𝗿𝗺𝗮𝘁𝘀 I also used ink in milk as a means of physical interaction with the diffusion pipeline. As I drop ink into milk, shapes come alive instantly. By learning to manipulate the combination of physical and digital elements, I steered the generated output toward my areas of interest. These iterations extended beyond ink in milk to include the format in which these elements are contained: a circular plate or a triptych of small stainless steel trays. These formats provide a structured yet flexible framework to explore themes and narratives across multiple visual spaces. It's magical. Some of that last iteration has been captured in this insightful article by Fast Company: #stablediffusion #realtime #ai

In the past few weeks, I deep dived into an exploration revolving around the use of physical interfaces to feed and interact with a real-time img2img diffusion pipeline using Stream Diffusion and SDXL Turbo. What really captivated me is to use my hands, objects, art supplies, tools, and light to create images and scenes. 𝗣𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗠𝗲𝗱𝗶𝗮 + 𝗧𝗼𝗼𝗹𝘀 I experimented with clay, manipulating different types and colors along with a selection of prompts. I used a magnifying glass, tracked in real-time, to focus the diffusion process on specific areas. Combining these tools created a dynamic and inspiring experience. Using magic clay to layer shapes and colors as a base for revealing landscapes and hidden worlds, and the magnifying glass to focus and reveal these details, was particularly effective. 𝗣𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗟𝗶𝗴𝗵𝘁 I used light as my method of interaction with the img2img diffusion. This approach felt special right away. There was something magical about holding a physical light source and seeing it influence the generated visuals. I iterated on this technique with themes like Rococo architecture, flowers, Brutalist architecture, hidden worlds, and origami landscapes. 𝗜𝗻𝗸 + 𝗛𝘆𝗯𝗿𝗶𝗱 𝗙𝗼𝗿𝗺𝗮𝘁𝘀 I also used ink in milk as a means of physical interaction with the diffusion pipeline. As I drop ink into milk, shapes come alive instantly. By learning to manipulate the combination of physical and digital elements, I steered the generated output toward my areas of interest. These iterations extended beyond ink in milk to include the format in which these elements are contained: a circular plate or a triptych of small stainless steel trays. These formats provide a structured yet flexible framework to explore themes and narratives across multiple visual spaces. It's magical. Some of that last iteration has been captured in this insightful article by Fast Company: #stablediffusion #realtime #ai

Hugues Bruyère

86,381 views • 2 years ago

Diffusion is the foundational ML framework behind state-of-the-art AI image and video generation, including Sora, Midjourney and Google Veo. In this episode of Decoded, Ankit Gupta sits down with Francois Chaubard to discuss how diffusion works, walk through a code sample, and explain why everyone training models should understand it. 00:00 Intro 00:33 What is diffusion? 02:50 What are applications of diffusion today? 04:06 Key innovations 07:01 Code examples 19:25 The "squint test" 22:27 Other areas diffusion is widely accessible 24:49 Outro

Diffusion is the foundational ML framework behind state-of-the-art AI image and video generation, including Sora, Midjourney and Google Veo. In this episode of Decoded, Ankit Gupta sits down with Francois Chaubard to discuss how diffusion works, walk through a code sample, and explain why everyone training models should understand it. 00:00 Intro 00:33 What is diffusion? 02:50 What are applications of diffusion today? 04:06 Key innovations 07:01 Code examples 19:25 The "squint test" 22:27 Other areas diffusion is widely accessible 24:49 Outro

Y Combinator

74,604 views • 7 months ago

Skinny Bob is back in the Cycle and we are being told he is 100% real alien DNA.. I put the original clips through my Diffusion process (Not Ai Upscale) to get the maximum detail.. I clip in a few interesting aspects of his features.. This is about as clean as i can get it while still respecting the original details in the video.

Skinny Bob is back in the Cycle and we are being told he is 100% real alien DNA.. I put the original clips through my Diffusion process (Not Ai Upscale) to get the maximum detail.. I clip in a few interesting aspects of his features.. This is about as clean as i can get it while still respecting the original details in the video.

ProPixel Video Analysis and Research

329,667 views • 1 year ago

A quick first pass at a new iteration of the realtime diffusion pipeline: physical interface → camera → realtime img2img → upscale → 3DGS through the new TripoSplat. Rough and held together with tape for now, but the whole loop runs end to end. Next: - better input and camera's pov, - splitting it across two machines to automate the hand-off, - and adding in the Tripo3D API for higher-res models... the 3D printer is just waiting for that. #diffusion #3DGS #TripoSplat

A quick first pass at a new iteration of the realtime diffusion pipeline: physical interface → camera → realtime img2img → upscale → 3DGS through the new TripoSplat. Rough and held together with tape for now, but the whole loop runs end to end. Next: - better input and camera's pov, - splitting it across two machines to automate the hand-off, - and adding in the Tripo3D API for higher-res models... the 3D printer is just waiting for that. #diffusion #3DGS #TripoSplat

Hugues Bruyère

15,526 views • 2 months ago

Wow, diffusion models (used in AI image generation) are also game engines - a type of world simulation. By predicting the next frame of the classic shooter DOOM, you get a playable game at 20 fps without any underlying real game engine. This video is from the diffusion model.

Wow, diffusion models (used in AI image generation) are also game engines - a type of world simulation. By predicting the next frame of the classic shooter DOOM, you get a playable game at 20 fps without any underlying real game engine. This video is from the diffusion model.

Ethan Mollick

1,769,195 views • 2 years ago

I got access to Gemini Diffusion for a few weeks. I believe this is one of the most important upcoming technologies in the AI space, it's like a GPT-4 moment all over again. Once the model gets better, it will completely reshape AI experiences for us. Video is in real time.

I got access to Gemini Diffusion for a few weeks. I believe this is one of the most important upcoming technologies in the AI space, it's like a GPT-4 moment all over again. Once the model gets better, it will completely reshape AI experiences for us. Video is in real time.

Pietro Schirano

109,187 views • 1 year ago

We are thrilled to announce "Layered Diffusion Brushes": a real-time training-free image editor powered by diffusion models. 🎨✨ This is new work from my PhD student Peyman Gholami Peyman Gholami. Explore the interactive demo and check out more videos at:

We are thrilled to announce "Layered Diffusion Brushes": a real-time training-free image editor powered by diffusion models. 🎨✨ This is new work from my PhD student Peyman Gholami Peyman Gholami. Explore the interactive demo and check out more videos at:

Robert Xiao

14,334 views • 2 years ago

I'm thrilled to announce the launch of ⚡️Flash Diffusion from Jasper! Earlier this year, with our acquisition of Clipdrop, we launched the Jasper AI Research Lab in Paris. Today, we are excited to release our first piece of groundbreaking research: the open-source distillation method, "Flash Diffusion". Flash Diffusion accelerates inference by 500%, reduces computing costs, and produces higher-quality image outputs. Dive into the details and discover how Flash Diffusion is set to revolutionize the field of AI and image synthesis. Read all about it here: Try a demo on Hugging Face:

I'm thrilled to announce the launch of ⚡️Flash Diffusion from Jasper! Earlier this year, with our acquisition of Clipdrop, we launched the Jasper AI Research Lab in Paris. Today, we are excited to release our first piece of groundbreaking research: the open-source distillation method, "Flash Diffusion". Flash Diffusion accelerates inference by 500%, reduces computing costs, and produces higher-quality image outputs. Dive into the details and discover how Flash Diffusion is set to revolutionize the field of AI and image synthesis. Read all about it here: Try a demo on Hugging Face:

Timothy Young

10,093 views • 2 years ago

Can finally explore latent space through feedback loops in real time with the new #streamdiffusion setup from Lyell. Being able to guide it with my mouse in real time is way too much fun

Can finally explore latent space through feedback loops in real time with the new #streamdiffusion setup from Lyell. Being able to guide it with my mouse in real time is way too much fun

Spencer Sterling

19,146 views • 1 year ago

As someone who builds with LLMs and diffusion models, this 14-minute heat equation video is the closest thing to a "why noise reversal actually works" explainer I've ever seen released for free. Everyone thinks diffusion is a novel AI concept. It's the heat equation from 1822. The forward process spreads noise like heat, and the reverse process is what your model learns. Bookmark & watch this weekend. Same math, from 19th-century physics to 2025 image generation.

As someone who builds with LLMs and diffusion models, this 14-minute heat equation video is the closest thing to a "why noise reversal actually works" explainer I've ever seen released for free. Everyone thinks diffusion is a novel AI concept. It's the heat equation from 1822. The forward process spreads noise like heat, and the reverse process is what your model learns. Bookmark & watch this weekend. Same math, from 19th-century physics to 2025 image generation.

fintex

15,604 views • 11 days ago

Today, we’re thrilled to announce the open weights for Stable Diffusion 3 Medium, the latest and most advanced text-to-image AI model in our Stable Diffusion 3 series! This new release represents a major milestone in the evolution of generative AI and continues our commitment to democratising this powerful technology. 🎉 Learn more and get started here:

Today, we’re thrilled to announce the open weights for Stable Diffusion 3 Medium, the latest and most advanced text-to-image AI model in our Stable Diffusion 3 series! This new release represents a major milestone in the evolution of generative AI and continues our commitment to democratising this powerful technology. 🎉 Learn more and get started here:

Stability AI

308,312 views • 2 years ago

Introducing SDXL Turbo: A real-time text-to-image generation model. SDXL Turbo achieves state-of-the-art performance with a new distillation technology, enabling single-step image generation with unprecedented quality, reducing the required step count from 50 to just one. The code, research paper, and weights for non-commercial use are now available on our website. You can test SDXL Turbo on Stability AI’s image editing platform Clipdrop, with a beta demonstration of the real-time text-to-image generation capabilities. Learn more:

Introducing SDXL Turbo: A real-time text-to-image generation model. SDXL Turbo achieves state-of-the-art performance with a new distillation technology, enabling single-step image generation with unprecedented quality, reducing the required step count from 50 to just one. The code, research paper, and weights for non-commercial use are now available on our website. You can test SDXL Turbo on Stability AI’s image editing platform Clipdrop, with a beta demonstration of the real-time text-to-image generation capabilities. Learn more:

Stability AI

976,610 views • 2 years ago

The rate of diffusion of this next generation of AI is unlike anything we've seen, but even more remarkable is the sense of empowerment it has already unlocked in every corner of the world, including rural India.

The rate of diffusion of this next generation of AI is unlike anything we've seen, but even more remarkable is the sense of empowerment it has already unlocked in every corner of the world, including rural India.

Satya Nadella

290,462 views • 3 years ago

The Waypoint-1.5 technical paper is now live. Waypoint-1.5 is a real-time video diffusion world model designed to run on consumer GPUs, bringing interactive world models closer to practical, accessible deployment.

The Waypoint-1.5 technical paper is now live. Waypoint-1.5 is a real-time video diffusion world model designed to run on consumer GPUs, bringing interactive world models closer to practical, accessible deployment.

Overworld

80,488 views • 1 month ago

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,657 views • 3 years ago

Mercury 2 is live 🚀🚀 The world’s first reasoning diffusion LLM, delivering 5x faster performance than leading speed-optimized LLMs. Watching the team turn years of research into a real product never gets old, and I’m incredibly proud of what we’ve built. We’re just getting started on what diffusion can do for language.

Mercury 2 is live 🚀🚀 The world’s first reasoning diffusion LLM, delivering 5x faster performance than leading speed-optimized LLMs. Watching the team turn years of research into a real product never gets old, and I’m incredibly proud of what we’ve built. We’re just getting started on what diffusion can do for language.

Stefano Ermon

1,043,651 views • 5 months ago

🥳We’re releasing StreamDiffusionV2 for the live-stream community—from individual creators with one GPU to enterprise platforms with many. StreamDiffusionV2 is our follow-up to StreamDiffusion: #StreamDiffusion powered real products, but temporal consistency still bugged us. We’re fixing that for the community: 🤔Efficiency ≠ live-readiness: many “real-time” video diffusion models are fast, but live streaming demands per-frame deadlines with low jitter. StreamDiffusionV2 balances temporal stability with strict per-frame deadlines, achieves fast interaction, and is ready for your AI Youtuber Stream Live👇 📒Project page: 💻Code:

🥳We’re releasing StreamDiffusionV2 for the live-stream community—from individual creators with one GPU to enterprise platforms with many. StreamDiffusionV2 is our follow-up to StreamDiffusion: #StreamDiffusion powered real products, but temporal consistency still bugged us. We’re fixing that for the community: 🤔Efficiency ≠ live-readiness: many “real-time” video diffusion models are fast, but live streaming demands per-frame deadlines with low jitter. StreamDiffusionV2 balances temporal stability with strict per-frame deadlines, achieves fast interaction, and is ready for your AI Youtuber Stream Live👇 📒Project page: 💻Code:

Chenfeng_X

20,424 views • 10 months ago

From painting to nearly real in just minutes. 🎨➡️👤 Krita Diffusion + FacePoke + Live Portrait + Runway Gen 3. This combination is taking us closer to reality faster than ever before. Is this the future of art, or is something lost in the speed? #AIArt #DigitalHumans #CreativeTech

From painting to nearly real in just minutes. 🎨➡️👤 Krita Diffusion + FacePoke + Live Portrait + Runway Gen 3. This combination is taking us closer to reality faster than ever before. Is this the future of art, or is something lost in the speed? #AIArt #DigitalHumans #CreativeTech

Martin Nebelong

73,099 views • 1 year ago