Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Say goodbye to green screens! DiffusionMat is an AI framework that uses a diffusion model to create refined alpha mattes by correcting the output at every denoising step to look more like the original input image.

dreamingtulpa's profile picture

Dreaming Tulpa 🥓👑

56,420 subscribers

146,582 Aufrufe • vor 2 Jahren •via X (Twitter)

KunstWissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Dreaming Tulpa 🥓👑
Dreaming Tulpa 🥓👑vor 2 Jahren

If you like Tweets like this, you might enjoy my weekly newsletter, #aiartweekly. A free, once–weekly e-mail round-up of the latest AI art news, interviews with artists and useful tools & resources. Join 2800+ subscribers here:

Profilbild von Thomas H. Chapin IV
Thomas H. Chapin IVvor 2 Jahren

Where’s the code though? All I’m seeing is a 404 page on the GitHub link?

Profilbild von Dreaming Tulpa 🥓👑
Dreaming Tulpa 🥓👑vor 2 Jahren

Works for me. They haven’t released the code yet though.

Profilbild von Sambhav Gupta
Sambhav Guptavor 2 Jahren

Gold for Video Editors i guess

Profilbild von (Alex) Compositing Academy
(Alex) Compositing Academyvor 2 Jahren

The examples show people’s hair over white backgrounds, and near stationary backgrounds. I don’t believe this at all until I see some more impressive demos.

Profilbild von A.I.Warper
A.I.Warpervor 2 Jahren

Installed everything then quickly realized..... there are no examples of how to even use the damn thing! What are the CLI inputs to get it working? lol

Profilbild von Dreaming Tulpa 🥓👑
Dreaming Tulpa 🥓👑vor 2 Jahren

Check under inference in the repo

Profilbild von Uncanny Harry AI
Uncanny Harry AIvor 2 Jahren

The white paper stuff dropping this week has been insane! Thanks for sharing it with us mate 👍

Profilbild von Dreaming Tulpa 🥓👑
Dreaming Tulpa 🥓👑vor 2 Jahren

Yeah, this week has been crazy

Profilbild von skidmarxist
skidmarxistvor 2 Jahren

goodbye

Ähnliche Videos

When creating ingredient images, try to create stylistically consistent images. If you use input images that are too disparate, the output may lead to a cut scene. For example, when generating a reference image, try using a prompt like: “Create the next scene in this storyboard, now show a roadside diner with the same look and feel as the original image.”
0:22

When creating ingredient images, try to create stylistically consistent images. If you use input images that are too disparate, the output may lead to a cut scene. For example, when generating a reference image, try using a prompt like: “Create the next scene in this storyboard, now show a roadside diner with the same look and feel as the original image.”

Google AI

23,526 Aufrufe • vor 10 Monaten

3DTopia-XL High-Quality 3D PBR Asset Generation via Primitive Diffusion demo: model: 3DTopia-XL scales high-quality 3D asset generation using Diffusion Transformer (DiT) built upon an expressive and efficient 3D representation, PrimX. The denoising process takes 5 seconds to generate a 3D PBR asset from text/image input which is ready for the graphics pipeline to use.
0:41

3DTopia-XL High-Quality 3D PBR Asset Generation via Primitive Diffusion demo: model: 3DTopia-XL scales high-quality 3D asset generation using Diffusion Transformer (DiT) built upon an expressive and efficient 3D representation, PrimX. The denoising process takes 5 seconds to generate a 3D PBR asset from text/image input which is ready for the graphics pipeline to use.

AK

87,282 Aufrufe • vor 1 Jahr

TurboEdit Instant text-based image editing discuss: We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.
0:22

TurboEdit Instant text-based image editing discuss: We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.

AK

16,062 Aufrufe • vor 2 Jahren

How can we help *any* image-input policy generalize better? 👉 Meet PEEK 🤖 — a framework that uses VLMs to decide *where* to look and *what* to do, so downstream policies — from ACT, 3D-DA, or even π₀ — generalize more effectively! 🧵
0:41

How can we help *any* image-input policy generalize better? 👉 Meet PEEK 🤖 — a framework that uses VLMs to decide *where* to look and *what* to do, so downstream policies — from ACT, 3D-DA, or even π₀ — generalize more effectively! 🧵

Jesse Zhang

13,789 Aufrufe • vor 10 Monaten

OpenAI President Greg Brockman on coming bi-directional voice models: "The obvious thing that you want to accomplish is a model in AI that works much more like you and I do... that's able to process input at the same time it's processing output.. It's going to be very, very exciting as you move to these natural, very human fluid-like conversational interfaces." At Big Technology AI Summit
2:34

OpenAI President Greg Brockman on coming bi-directional voice models: "The obvious thing that you want to accomplish is a model in AI that works much more like you and I do... that's able to process input at the same time it's processing output.. It's going to be very, very exciting as you move to these natural, very human fluid-like conversational interfaces." At Big Technology AI Summit

Alex Kantrowitz

21,614 Aufrufe • vor 2 Monaten

🖼️🎞️🔊📄Excited to introduce Composable Diffusion (CoDi), a new generative-AI foundation model that can take any combo of input modalities & generate any combo of output modalities (text, audio, image, video)! Ziyi Yang Chenguang Zhu Mohit Bansal 🧵👇 #CoDi
0:28

🖼️🎞️🔊📄Excited to introduce Composable Diffusion (CoDi), a new generative-AI foundation model that can take any combo of input modalities & generate any combo of output modalities (text, audio, image, video)! Ziyi Yang Chenguang Zhu Mohit Bansal 🧵👇 #CoDi

Zineng Tang

105,290 Aufrufe • vor 3 Jahren

Imagine a world where every image has a ZK verified checkmark, proving that the image came from an authentic camera, not generated by an AI model. threadguy wants to know, "Who's building this? I want to invest."
1:38

Imagine a world where every image has a ZK verified checkmark, proving that the image came from an authentic camera, not generated by an AI model. threadguy wants to know, "Who's building this? I want to invest."

Succinct

19,382 Aufrufe • vor 1 Jahr

What you’re seeing Is the truffle learning To express itself Through light Every input (user keystroke, prompt, query) Produces embeddings That are live-diffused To the LED’s The truffle runs a diffusion model In parallel To express itself to the world
0:26

What you’re seeing Is the truffle learning To express itself Through light Every input (user keystroke, prompt, query) Produces embeddings That are live-diffused To the LED’s The truffle runs a diffusion model In parallel To express itself to the world

simp 4 satoshi

19,542 Aufrufe • vor 10 Monaten

Did you know you can create captivating motion sync videos on Yapper Follow the steps below and push your creativity boundaries Step 1: Go to on your browser step 2: Click on videos and select Motion Sync Step 3: Select Base Video The video's movement will be used to animate the reference image. Step 4: Add Reference Image Upload an image of a character to be animated by the base video. Step 5: Describe Your Vision Write a prompt that guides the background of the output video. Step 6: Generate Select your model and the output video will be generated in real-time
1:56

Did you know you can create captivating motion sync videos on Yapper Follow the steps below and push your creativity boundaries Step 1: Go to on your browser step 2: Click on videos and select Motion Sync Step 3: Select Base Video The video's movement will be used to animate the reference image. Step 4: Add Reference Image Upload an image of a character to be animated by the base video. Step 5: Describe Your Vision Write a prompt that guides the background of the output video. Step 6: Generate Select your model and the output video will be generated in real-time

Kuria | AI

46,344 Aufrufe • vor 3 Monaten

From LA to New York, the model continuously impresses me. This time utilizing an Image To World, the consistency in aesthetics from the input image is remarkable.
0:57

From LA to New York, the model continuously impresses me. This time utilizing an Image To World, the consistency in aesthetics from the input image is remarkable.

Theoretically Media

34,467 Aufrufe • vor 6 Monaten

You imagine it. We create it. 🎨 Introducing #PixelStudio, a first-of-its-kind image generator on #Pixel9 powered by an on-device diffusion model¹ running on Tensor G4 and our Imagen 3 text-to-image model in the cloud.
0:34

You imagine it. We create it. 🎨 Introducing #PixelStudio, a first-of-its-kind image generator on #Pixel9 powered by an on-device diffusion model¹ running on Tensor G4 and our Imagen 3 text-to-image model in the cloud.

Made by Google

28,283 Aufrufe • vor 2 Jahren

Google researchers just developed GameNGen, an AI that can simulate DOOM at over 20 fps It works by predicting each frame in real time with a diffusion model At scale this could mean AI will be able to create games on the fly, personalized to each player
1:22

Google researchers just developed GameNGen, an AI that can simulate DOOM at over 20 fps It works by predicting each frame in real time with a diffusion model At scale this could mean AI will be able to create games on the fly, personalized to each player

Rowan Cheung

880,403 Aufrufe • vor 2 Jahren

Stable Diffusion 3.5 from Stability AI is now LIVE on Civitai! Stable Diffusion 3.5 Large is a Multimodal Diffusion Transformer text-to-image model that features improved performance in image quality, typography, complex prompt understanding, and resource-efficiency. Generate with SD 3.5 Live on Civitai here👇🏽 All images used to create this video were generated with SD 3.5 by our community 🫶🏽
0:25

Stable Diffusion 3.5 from Stability AI is now LIVE on Civitai! Stable Diffusion 3.5 Large is a Multimodal Diffusion Transformer text-to-image model that features improved performance in image quality, typography, complex prompt understanding, and resource-efficiency. Generate with SD 3.5 Live on Civitai here👇🏽 All images used to create this video were generated with SD 3.5 by our community 🫶🏽

Civitai

31,428 Aufrufe • vor 1 Jahr

Through the length of SDXL-Turbo you can see the image forming in the activations like an object taking shape in the afternoon clouds. Like David emerging from a block of granite. At two-thirds through the model you can see the image clearly. It's spatially crisp, refined. Then in the last third of the model something strange but predictable happens: the model translates this well-formed image back into noise. By the output of the U-net, the activations are again completely uninterpretable. Why does this happen? Because we're asking the model to predict the noise rather than the clean image. This is awkward. Like Michelangelo visualizing the chips of granite surrounding the David rather than the David itself. Like asking a friend to look at a cloud and imagine not a shape, but the extra bits of cloud that when removed will leave a shape remaining.
1:22

Through the length of SDXL-Turbo you can see the image forming in the activations like an object taking shape in the afternoon clouds. Like David emerging from a block of granite. At two-thirds through the model you can see the image clearly. It's spatially crisp, refined. Then in the last third of the model something strange but predictable happens: the model translates this well-formed image back into noise. By the output of the U-net, the activations are again completely uninterpretable. Why does this happen? Because we're asking the model to predict the noise rather than the clean image. This is awkward. Like Michelangelo visualizing the chips of granite surrounding the David rather than the David itself. Like asking a friend to look at a cloud and imagine not a shape, but the extra bits of cloud that when removed will leave a shape remaining.

Rudy Gilman

11,641 Aufrufe • vor 1 Jahr

This may look like a game of Counterstrike running slowly, but it actually extraordinary. The entire game is created, frame-by-frame, on my *home computer* by an AI diffusion model in response to my actions. There is no game engine, just a "world model" trained on Counterstrike
0:30

This may look like a game of Counterstrike running slowly, but it actually extraordinary. The entire game is created, frame-by-frame, on my *home computer* by an AI diffusion model in response to my actions. There is no game engine, just a "world model" trained on Counterstrike

Ethan Mollick

376,399 Aufrufe • vor 1 Jahr

Excited to share Muse Spark! It's a strong natively multimodal model with many surprising properties that emerged. Here, the model is able to use Python tools to make a playable Sudoku game on the web from an image input of the board. ✨
0:42

Excited to share Muse Spark! It's a strong natively multimodal model with many surprising properties that emerged. Here, the model is able to use Python tools to make a playable Sudoku game on the web from an image input of the board. ✨

Matt Deitke

29,581 Aufrufe • vor 4 Monaten

Microsoft published an image-to-GIF model called Pix2Gif! They claim that the model is better at understand motion, while we’re not talking Sora levels here, it’s certainly a step up compared to the slow-motion outputs we’re currently used to.
0:27

Microsoft published an image-to-GIF model called Pix2Gif! They claim that the model is better at understand motion, while we’re not talking Sora levels here, it’s certainly a step up compared to the slow-motion outputs we’re currently used to.

Dreaming Tulpa 🥓👑

14,090 Aufrufe • vor 2 Jahren

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.
0:36

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

AK

143,469 Aufrufe • vor 3 Jahren