Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Can you make a jigsaw puzzle with two different solutions? Or an image that changes appearance when flipped? We can do that, and a lot more, by using diffusion models to generate optical illusions! Continue reading for more illusions and method details 🧵

Daniel Geng

1,504 subscribers

125,886 views • 2 years ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Can we use motion to prompt diffusion models? Our #ICLR2024 paper does just that. We propose Motion Guidance, a technique that allows users to edit an image by specifying “where things should move.”

Can we use motion to prompt diffusion models? Our #ICLR2024 paper does just that. We propose Motion Guidance, a technique that allows users to edit an image by specifying “where things should move.”

Daniel Geng

16,306 views • 2 years ago

Want to generate images from the world's best models like Nano Banana or GPT Image side by side? Now you can with Image Lab 🚀 Run a single prompt across up to 25 models, with up to 20 images from each, and see results in seconds. You've seen our leaderboards. Now generate and evaluate the models yourself. Link in thread 🧵👇

Want to generate images from the world's best models like Nano Banana or GPT Image side by side? Now you can with Image Lab 🚀 Run a single prompt across up to 25 models, with up to 20 images from each, and see results in seconds. You've seen our leaderboards. Now generate and evaluate the models yourself. Link in thread 🧵👇

Artificial Analysis

12,250 views • 6 months ago

Multi-bot chat allows you to seamlessly combine the unique strengths of different bots to complete tasks. For example, using fal’s latest bots, you can generate an image and remove the background within one conversational thread for a more efficient workflow. (1/2)

Multi-bot chat allows you to seamlessly combine the unique strengths of different bots to complete tasks. For example, using fal’s latest bots, you can generate an image and remove the background within one conversational thread for a more efficient workflow. (1/2)

Poe

19,538 views • 2 years ago

3. Modify image Use this prompt to modify the image you have chosen: "Modify image [1] with seed [1470033597]: add a parrot on her shoulder" Dall-E 3 identifies the image and makes the changes for you! Tips and tricks: - You can generate as many variations as you like. - It's also possible to remove an element from the image using the same method. - Sometimes the image isn't 100% identical, but still looks very similar.

3. Modify image Use this prompt to modify the image you have chosen: "Modify image [1] with seed [1470033597]: add a parrot on her shoulder" Dall-E 3 identifies the image and makes the changes for you! Tips and tricks: - You can generate as many variations as you like. - It's also possible to remove an element from the image using the same method. - Sometimes the image isn't 100% identical, but still looks very similar.

Paul Couvert

44,073 views • 2 years ago

DINOv2 by Meta AI is the first method for training computer vision models that uses self-supervised learning to achieve results matching or exceeding industry standards. More details and a demo ⬇️

DINOv2 by Meta AI is the first method for training computer vision models that uses self-supervised learning to achieve results matching or exceeding industry standards. More details and a demo ⬇️

AI at Meta

96,119 views • 3 years ago

when you get access to gpt-5, try a message like "use beatbot to make a sick beat to celebrate gpt-5". it's a nice preview of what we think this will be like as AI starts to generate its own UX and interfaces get more dynamic. it's cool that you can interact with the synthesizer directly or ask chatgpt to make changes!

when you get access to gpt-5, try a message like "use beatbot to make a sick beat to celebrate gpt-5". it's a nice preview of what we think this will be like as AI starts to generate its own UX and interfaces get more dynamic. it's cool that you can interact with the synthesizer directly or ask chatgpt to make changes!

Sam Altman

2,529,240 views • 1 year ago

I topped up $5 on an API aggregator ToAPIs Then I found out GPT Image 2 costs only around $0.015 per image. If you do a lot of testing or batch-generate commercial AI images, that difference adds up fast. I think I just found the secret to generating more, testing more, and spending less. And it’s not just one model. With the same key, you can access 50+ models for image, video, and text, including GPT Image 2, Gemini Omni, Seedance 2.0, Kling AI 3.0, grok-video-1.5-preview, and more. Some models are priced up to 80% lower than official platforms. Just top up and test what you need: Made on ToAPIs with GPT Image 2 + Seedance 2.0

I topped up $5 on an API aggregator ToAPIs Then I found out GPT Image 2 costs only around $0.015 per image. If you do a lot of testing or batch-generate commercial AI images, that difference adds up fast. I think I just found the secret to generating more, testing more, and spending less. And it’s not just one model. With the same key, you can access 50+ models for image, video, and text, including GPT Image 2, Gemini Omni, Seedance 2.0, Kling AI 3.0, grok-video-1.5-preview, and more. Some models are priced up to 80% lower than official platforms. Just top up and test what you need: Made on ToAPIs with GPT Image 2 + Seedance 2.0

Shami

22,991 views • 2 months ago

Introducing ⇆ Marigold-DC — our training-free zero-shot approach to monocular Depth Completion with guided diffusion! If you have ever wondered how else a long denoising diffusion schedule can be useful, we have an answer for you! Details 🧵

Introducing ⇆ Marigold-DC — our training-free zero-shot approach to monocular Depth Completion with guided diffusion! If you have ever wondered how else a long denoising diffusion schedule can be useful, we have an answer for you! Details 🧵

Anton Obukhov

39,324 views • 1 year ago

LLMs require more GPU memory as they generate longer responses. Can we make GPU memory constant without significantly sacrificing accuracy? IceCache is a new method for managing KV caches that leverages Dynamic Continuous Indexing (DCI) to efficiently group and retrieve tokens by semantics. Joint work w/ Yuzhen Mao, Qitong Wang and Martin Ester. For details, check out the links below.

LLMs require more GPU memory as they generate longer responses. Can we make GPU memory constant without significantly sacrificing accuracy? IceCache is a new method for managing KV caches that leverages Dynamic Continuous Indexing (DCI) to efficiently group and retrieve tokens by semantics. Joint work w/ Yuzhen Mao, Qitong Wang and Martin Ester. For details, check out the links below.

Ke Li 🍁

21,163 views • 4 months ago

Chop the gradients ✂️! We found that truncating decoder gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.

Chop the gradients ✂️! We found that truncating decoder gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.

Felix Heide

28,399 views • 4 months ago

The Hidden Language of Diffusion Models paper page: tackle the challenge of understanding concept representations in text-to-image models by decomposing an input text prompt into a small set of interpretable elements. This is achieved by learning a pseudo-token that is a sparse weighted combination of tokens from the model's vocabulary, with the objective of reconstructing the images generated for the given concept. Applied over the state-of-the-art Stable Diffusion model, this decomposition reveals non-trivial and surprising structures in the representations of concepts. For example, we find that some concepts such as "a president" or "a composer" are dominated by specific instances (e.g., "Obama", "Biden") and their interpolations. Other concepts, such as "happiness" combine associated terms that can be concrete ("family", "laughter") or abstract ("friendship", "emotion"). In addition to peering into the inner workings of Stable Diffusion, our method also enables applications such as single-image decomposition to tokens, bias detection and mitigation, and semantic image manipulation

The Hidden Language of Diffusion Models paper page: tackle the challenge of understanding concept representations in text-to-image models by decomposing an input text prompt into a small set of interpretable elements. This is achieved by learning a pseudo-token that is a sparse weighted combination of tokens from the model's vocabulary, with the objective of reconstructing the images generated for the given concept. Applied over the state-of-the-art Stable Diffusion model, this decomposition reveals non-trivial and surprising structures in the representations of concepts. For example, we find that some concepts such as "a president" or "a composer" are dominated by specific instances (e.g., "Obama", "Biden") and their interpolations. Other concepts, such as "happiness" combine associated terms that can be concrete ("family", "laughter") or abstract ("friendship", "emotion"). In addition to peering into the inner workings of Stable Diffusion, our method also enables applications such as single-image decomposition to tokens, bias detection and mitigation, and semantic image manipulation

AK

41,830 views • 3 years ago

You can now turn any image on X into a video. Long press on an image that you like and select Make Video with Grok.

You can now turn any image on X into a video. Long press on an image that you like and select Make Video with Grok.

Chaitu

993,765 views • 1 year ago

1/ Happy to share VADER: Video Diffusion Alignment via Reward Gradients. We adapt foundational video diffusion models using pre-trained reward models to generate high-quality, aligned videos for various end-applications. Below we generated a short movie using VADER 😀, we used ChatGPT to write a script and an off-the-shelf AI music generator to generate the sound. Our code & weights are open-sourced:

1/ Happy to share VADER: Video Diffusion Alignment via Reward Gradients. We adapt foundational video diffusion models using pre-trained reward models to generate high-quality, aligned videos for various end-applications. Below we generated a short movie using VADER 😀, we used ChatGPT to write a script and an off-the-shelf AI music generator to generate the sound. Our code & weights are open-sourced:

Mihir Prabhudesai

13,403 views • 2 years ago

Generate mermaid diagrams that you can then edit manually, using your favorite coding agent! Just use Decode's MCP server and ask your agent to explain things or help you think more clearly when designing & planning.

Generate mermaid diagrams that you can then edit manually, using your favorite coding agent! Just use Decode's MCP server and ask your agent to explain things or help you think more clearly when designing & planning.

Francois Laberge

35,959 views • 3 months ago

If you’re working a 9-to-5 and a side hustle, start with diet and exercise. Even if you can’t hit the gym, stop three or four times a day and do as many push-ups as you can. Do that for three months, and you’ll see real changes. That progress will motivate you to work out more and eventually hit the gym.

If you’re working a 9-to-5 and a side hustle, start with diet and exercise. Even if you can’t hit the gym, stop three or four times a day and do as many push-ups as you can. Do that for three months, and you’ll see real changes. That progress will motivate you to work out more and eventually hit the gym.

JWaller7

75,528 views • 1 year ago

Ever watch an agent solve a problem with 100% confidence and get it completely wrong? You can fix it… if you catch it. But then you need to watch every… single… run. New paper: We show that a meta-agent can do this for you, so your agents get ~2x more done on their own. 🧵

Ever watch an agent solve a problem with 100% confidence and get it completely wrong? You can fix it… if you catch it. But then you need to watch every… single… run. New paper: We show that a meta-agent can do this for you, so your agents get ~2x more done on their own. 🧵

Weiyan Shi

63,085 views • 1 month ago

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,062 views • 1 year ago

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:

Chenfeng_X

28,308 views • 1 year ago