Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Dreamix: Video Diffusion Models are General Video Editors abs: project page: present diffusion-based method that is able to perform text-based motion and appearance editing of general videos

_akhaliq's profile picture

AK

536,226 subscribers

398,258 Aufrufe • vor 3 Jahren •via X (Twitter)

BildungGesundheit & WellnessWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Abi
Abivor 3 Jahren

pre: "You won't believe your eyes!" post: "You can't believe your eyes!"

Profilbild von Rekt Adult (🩸,🩸)
Rekt Adult (🩸,🩸)vor 3 Jahren

where/when/how can someone use it? they didn't include the code with the github page

Profilbild von Minute Movies
Minute Moviesvor 3 Jahren

excuse me what

Profilbild von William Lamkin
William Lamkinvor 3 Jahren

Nice find. 3D + Motion and editing tools seem to be the subject of the next wave of papers behind audio-based GenAI models

Profilbild von Stern - uɹǝʇS
Stern - uɹǝʇSvor 3 Jahren

@CoffeeVectors @TomLikesRobots @GanWeaving as soon as models of this fidelity ad coherence arrive publicly …… wow

Profilbild von Jason Murphy
Jason Murphyvor 3 Jahren

Amazing!

Profilbild von Vic J
Vic Jvor 3 Jahren

@OliverLaufer

Profilbild von Bernard Bontemps
Bernard Bontempsvor 3 Jahren

@Francescu

Profilbild von Leo Enin 🍰
Leo Enin 🍰vor 3 Jahren

Absolutely astonishing! Imagine graphic tools 10 years from now. Now imagine video games and movies in 20 years, given the progress we got over the last 20. And I'm not even talking about other industries here. The world is rapidly changing, and it's beautiful.

Profilbild von Guilherme
Guilhermevor 3 Jahren

@BaixaEssaPorra

Ähnliche Videos

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.
0:36

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

AK

143,557 Aufrufe • vor 3 Jahren

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models turn the publicly available, state-of-the-art text-to-image LDM Stable Diffusion into an efficient and expressive text-to-video model with resolution up to 1280 x 2048 abs: project page:
1:38

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models turn the publicly available, state-of-the-art text-to-image LDM Stable Diffusion into an efficient and expressive text-to-video model with resolution up to 1280 x 2048 abs: project page:

AK

718,894 Aufrufe • vor 3 Jahren

Single Motion Diffusion abs: project page:
2:15

Single Motion Diffusion abs: project page:

AK

72,789 Aufrufe • vor 3 Jahren

Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models abs: project page:
0:42

Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models abs: project page:

AK

121,020 Aufrufe • vor 3 Jahren

We present TeSMo ( ), a text-controlled scene-aware motion generation method based on denoising diffusion models. It’s an exciting collaboration with Justus Thies, Michael Black, Jason Peng, Davis Rempe. (1/7)
0:22

We present TeSMo ( ), a text-controlled scene-aware motion generation method based on denoising diffusion models. It’s an exciting collaboration with Justus Thies, Michael Black, Jason Peng, Davis Rempe. (1/7)

Hongwei Yi

45,744 Aufrufe • vor 2 Jahren

Introducing 🔥BLIP-Diffusion🔥, a novel method for enabling Text-to-image Diffusion models with multimodal controllable generation/editing, powered by BLIP-2 pre-trained text-aligned subject representation. Paper: Project: (1/n)
2:03

Introducing 🔥BLIP-Diffusion🔥, a novel method for enabling Text-to-image Diffusion models with multimodal controllable generation/editing, powered by BLIP-2 pre-trained text-aligned subject representation. Paper: Project: (1/n)

Steven Hoi

40,615 Aufrufe • vor 3 Jahren

🏃Motion Tokenizer Bridging Semantic and Kinematic Conditions🏃‍♀️ #MoTok is a diffusion-based tokenizer that unifies *perception-planning-control*, combining the strengths of continuous diffusion & discrete tokens - Project: - Code:
5:01

🏃Motion Tokenizer Bridging Semantic and Kinematic Conditions🏃‍♀️ #MoTok is a diffusion-based tokenizer that unifies *perception-planning-control*, combining the strengths of continuous diffusion & discrete tokens - Project: - Code:

Ziwei Liu

19,375 Aufrufe • vor 7 Monaten

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:
0:24

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:

AK

304,708 Aufrufe • vor 3 Jahren

Energy-Based Models can now scale better than Diffusion models. Introducing TEM: Transformer Energy-based Models TLDR: • TEM outscales Diffusion without guidance • First scalable contrastive Energy-Based Models • Better generalization, test-time scaling, and OOD detection than Diffusion 🧵
1:08

Energy-Based Models can now scale better than Diffusion models. Introducing TEM: Transformer Energy-based Models TLDR: • TEM outscales Diffusion without guidance • First scalable contrastive Energy-Based Models • Better generalization, test-time scaling, and OOD detection than Diffusion 🧵

Alexi Gladstone

88,302 Aufrufe • vor 7 Tagen

Scaling up GANs for Text-to-Image Synthesis present our 1B-parameter GigaGAN, achieving lower FID than Stable Diffusion v1.5, DALL·E 2, and Parti-750M. It generates 512px outputs at 0.13s, orders of magnitude faster than diffusion and autoregressive models, and inherits the disentangled, continuous, and controllable latent space of GANs abs: project page:
1:18

Scaling up GANs for Text-to-Image Synthesis present our 1B-parameter GigaGAN, achieving lower FID than Stable Diffusion v1.5, DALL·E 2, and Parti-750M. It generates 512px outputs at 0.13s, orders of magnitude faster than diffusion and autoregressive models, and inherits the disentangled, continuous, and controllable latent space of GANs abs: project page:

AK

278,220 Aufrufe • vor 3 Jahren

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:
0:46

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:

Yuan Liu

33,490 Aufrufe • vor 1 Jahr

Today we’re sharing two new advances in our generative AI research: Emu Video & Emu Edit. Details ➡️ These new models deliver exciting results in high quality, diffusion-based text-to-video generation & controlled image editing w/ text instructions. 🧵
0:33

Today we’re sharing two new advances in our generative AI research: Emu Video & Emu Edit. Details ➡️ These new models deliver exciting results in high quality, diffusion-based text-to-video generation & controlled image editing w/ text instructions. 🧵

AI at Meta

798,500 Aufrufe • vor 3 Jahren

PAIR-Diffusion: Object-Level Image Editing with Structure-and-Appearance Paired Diffusion Models Gradio demo is out on Hugging Face Spaces demo:
1:05

PAIR-Diffusion: Object-Level Image Editing with Structure-and-Appearance Paired Diffusion Models Gradio demo is out on Hugging Face Spaces demo:

AK

87,852 Aufrufe • vor 3 Jahren

I finally released my new video on YouTube about Diffusion Models / Score-Based Generative Models. Literally planned this for a year and put so much work in. I think this approach to diffusion models is so intuitive and highly recommend giving that a go! Video is 38min long, so you will need some time to watch that haha.
0:59

I finally released my new video on YouTube about Diffusion Models / Score-Based Generative Models. Literally planned this for a year and put so much work in. I think this approach to diffusion models is so intuitive and highly recommend giving that a go! Video is 38min long, so you will need some time to watch that haha.

dome | Outlier

54,611 Aufrufe • vor 2 Jahren

Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model abs: project page:
2:16

Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model abs: project page:

AK

90,089 Aufrufe • vor 3 Jahren

A new frontier. Project Starlight—the first and only diffusion-based AI model for enhancing video. Launching soon.
0:22

A new frontier. Project Starlight—the first and only diffusion-based AI model for enhancing video. Launching soon.

Topaz Labs

22,375 Aufrufe • vor 1 Jahr

StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis significantly improves over previous GANs and outperforms distilled diffusion models in terms of sample quality and speed abs: project page:
2:18

StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis significantly improves over previous GANs and outperforms distilled diffusion models in terms of sample quality and speed abs: project page:

AK

425,803 Aufrufe • vor 3 Jahren

Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models paper page: Traditional 3D content creation tools empower users to bring their imagination to life by giving them direct control over a scene's geometry, appearance, motion, and camera path. Creating computer-generated videos, however, is a tedious manual process, which can be automated by emerging text-to-video diffusion models. Despite great promise, video diffusion models are difficult to control, hindering a user to apply their own creativity rather than amplifying it. To address this challenge, we present a novel approach that combines the controllability of dynamic 3D meshes with the expressivity and editability of emerging diffusion models. For this purpose, our approach takes an animated, low-fidelity rendered mesh as input and injects the ground truth correspondence information obtained from the dynamic mesh into various stages of a pre-trained text-to-image generation model to output high-quality and temporally consistent frames. We demonstrate our approach on various examples where motion can be obtained by animating rigged assets or changing the camera path.
0:23

Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models paper page: Traditional 3D content creation tools empower users to bring their imagination to life by giving them direct control over a scene's geometry, appearance, motion, and camera path. Creating computer-generated videos, however, is a tedious manual process, which can be automated by emerging text-to-video diffusion models. Despite great promise, video diffusion models are difficult to control, hindering a user to apply their own creativity rather than amplifying it. To address this challenge, we present a novel approach that combines the controllability of dynamic 3D meshes with the expressivity and editability of emerging diffusion models. For this purpose, our approach takes an animated, low-fidelity rendered mesh as input and injects the ground truth correspondence information obtained from the dynamic mesh into various stages of a pre-trained text-to-image generation model to output high-quality and temporally consistent frames. We demonstrate our approach on various examples where motion can be obtained by animating rigged assets or changing the camera path.

AK

117,091 Aufrufe • vor 2 Jahren

We present Neural Riemannian Motion Fields (NRMF) as a promising paradigm for generative modeling of articulated motion. Unlike various diffusion based models, NRMF is useful both in generation and in deployment as a prior across tasks: #CVPR #AI #CVML👇
3:59

We present Neural Riemannian Motion Fields (NRMF) as a promising paradigm for generative modeling of articulated motion. Unlike various diffusion based models, NRMF is useful both in generation and in deployment as a prior across tasks: #CVPR #AI #CVML👇

Tolga Birdal

16,795 Aufrufe • vor 1 Jahr