Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Struggling with slow inference of diffusion and flow models? Check out the video below—I’ve been using our new FastGen library to achieve 7-28x acceleration for text-2-image and {text,image,video}-2-video generation without sacrificing visual fidelity!

julberner's profile picture

Julius Berner

1,299 subscribers

13,742 Aufrufe • vor 6 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

We have released an update for both text to video and image to video generation with Gen-2, bringing major improvements to both the fidelity and consistency of video results. Try it now at
1:15

We have released an update for both text to video and image to video generation with Gen-2, bringing major improvements to both the fidelity and consistency of video results. Try it now at

Runway

601,457 Aufrufe • vor 2 Jahren

(1/2) LightIt: Illumination Modeling and Control for Diffusion Models! #CVPR2024 We facilitate lighting control for novel image generation from text prompts. We can also edit lighting for a given input image. Video: Project:
0:31

(1/2) LightIt: Illumination Modeling and Control for Diffusion Models! #CVPR2024 We facilitate lighting control for novel image generation from text prompts. We can also edit lighting for a given input image. Video: Project:

Matthias Niessner

19,849 Aufrufe • vor 2 Jahren

New Kling, Runway, Luma competitor? Open text and image video generation model Pyramidal Flow Matching for Efficient Video Generative Modeling
2:23

New Kling, Runway, Luma competitor? Open text and image video generation model Pyramidal Flow Matching for Efficient Video Generative Modeling

AK

91,250 Aufrufe • vor 1 Jahr

Compute is the backbone of the AI-driven future. OptimAI Compute Engine would enable scalable, high-performance workloads across image and video models, supporting: • Text-to-image generation • Image editing and inpainting • 2K / 4K / 8K super-resolution • Brand-aligned visual synthesis • OCR and image intelligence • Video generation and motion synthesis • Frame interpolation and enhancement • Multimodal model inference Distributed compute built for faster inference, shorter training cycles, and production-grade AI execution.
0:24

Compute is the backbone of the AI-driven future. OptimAI Compute Engine would enable scalable, high-performance workloads across image and video models, supporting: • Text-to-image generation • Image editing and inpainting • 2K / 4K / 8K super-resolution • Brand-aligned visual synthesis • OCR and image intelligence • Video generation and motion synthesis • Frame interpolation and enhancement • Multimodal model inference Distributed compute built for faster inference, shorter training cycles, and production-grade AI execution.

OptimAI Network

18,855 Aufrufe • vor 6 Monaten

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models turn the publicly available, state-of-the-art text-to-image LDM Stable Diffusion into an efficient and expressive text-to-video model with resolution up to 1280 x 2048 abs: project page:
1:38

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models turn the publicly available, state-of-the-art text-to-image LDM Stable Diffusion into an efficient and expressive text-to-video model with resolution up to 1280 x 2048 abs: project page:

AK

718,760 Aufrufe • vor 3 Jahren

New text and image to video generation AI model Open-Sora-Plan-v1.3.0
1:37

New text and image to video generation AI model Open-Sora-Plan-v1.3.0

AK

51,843 Aufrufe • vor 1 Jahr

Today we’re sharing two new advances in our generative AI research: Emu Video & Emu Edit. Details ➡️ These new models deliver exciting results in high quality, diffusion-based text-to-video generation & controlled image editing w/ text instructions. 🧵
0:33

Today we’re sharing two new advances in our generative AI research: Emu Video & Emu Edit. Details ➡️ These new models deliver exciting results in high quality, diffusion-based text-to-video generation & controlled image editing w/ text instructions. 🧵

AI at Meta

798,304 Aufrufe • vor 2 Jahren

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.
0:36

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

AK

143,469 Aufrufe • vor 3 Jahren

Make Pixels Dance: High-Dynamic Video Generation paper page: Creating high-dynamic videos such as motion-rich actions and sophisticated visual effects poses a significant challenge in the field of artificial intelligence. Unfortunately, current state-of-the-art video generation methods, primarily focusing on text-to-video generation, tend to produce video clips with minimal motions despite maintaining high fidelity. We argue that relying solely on text instructions is insufficient and suboptimal for video generation. In this paper, we introduce PixelDance, a novel approach based on diffusion models that incorporates image instructions for both the first and last frames in conjunction with text instructions for video generation. Comprehensive experimental results demonstrate that PixelDance trained with public data exhibits significantly better proficiency in synthesizing videos with complex scenes and intricate motions, setting a new standard for video generation.
3:09

Make Pixels Dance: High-Dynamic Video Generation paper page: Creating high-dynamic videos such as motion-rich actions and sophisticated visual effects poses a significant challenge in the field of artificial intelligence. Unfortunately, current state-of-the-art video generation methods, primarily focusing on text-to-video generation, tend to produce video clips with minimal motions despite maintaining high fidelity. We argue that relying solely on text instructions is insufficient and suboptimal for video generation. In this paper, we introduce PixelDance, a novel approach based on diffusion models that incorporates image instructions for both the first and last frames in conjunction with text instructions for video generation. Comprehensive experimental results demonstrate that PixelDance trained with public data exhibits significantly better proficiency in synthesizing videos with complex scenes and intricate motions, setting a new standard for video generation.

AK

102,706 Aufrufe • vor 2 Jahren

Introducing Poe Apps: a new, easy way to create and use visual interfaces into any combination of the 100+ text, image, video, and audio models on Poe. (1/5)
0:54

Introducing Poe Apps: a new, easy way to create and use visual interfaces into any combination of the 100+ text, image, video, and audio models on Poe. (1/5)

Poe

66,858 Aufrufe • vor 1 Jahr

👀 Pixel perfect 💎✨ 🖼️ Edify Image from #NVIDIAResearch is a family of diffusion models that supports a wide range of applications, including text-to-image synthesis, 4K upsampling, ControlNets, 360° HDR panorama generation, and finetuning for image customization. 🧵 1/2
2:48

👀 Pixel perfect 💎✨ 🖼️ Edify Image from #NVIDIAResearch is a family of diffusion models that supports a wide range of applications, including text-to-image synthesis, 4K upsampling, ControlNets, 360° HDR panorama generation, and finetuning for image customization. 🧵 1/2

NVIDIA AI Developer

14,747 Aufrufe • vor 1 Jahr

ByteDance just unveiled HuMo, a unified framework for human-centric video generation. With HuMo you can: • Generate videos from text + image • Create audio-synced videos from text + audio • Combine text, image, and audio for maximum control • Preserve consistent subjects across frames • Achieve natural audio-visual synchronization This isn’t just another release , it’s a big step toward fully controllable, coherent, and cinematic AI video.
1:51

ByteDance just unveiled HuMo, a unified framework for human-centric video generation. With HuMo you can: • Generate videos from text + image • Create audio-synced videos from text + audio • Combine text, image, and audio for maximum control • Preserve consistent subjects across frames • Achieve natural audio-visual synchronization This isn’t just another release , it’s a big step toward fully controllable, coherent, and cinematic AI video.

DStudioproject

29,408 Aufrufe • vor 11 Monaten

Ray2 Keyframes, Extend & Loop is now here. Create your visual story with precise frame-by-frame control, seamless transitions, spatial exploration and long-form video durations using #Ray2 Text-to-Video and Image-to-Video models in Dream Machine.
0:38

Ray2 Keyframes, Extend & Loop is now here. Create your visual story with precise frame-by-frame control, seamless transitions, spatial exploration and long-form video durations using #Ray2 Text-to-Video and Image-to-Video models in Dream Machine.

Luma

7,362,407 Aufrufe • vor 1 Jahr

Text is often the hardest part of image generation to get right. MAI-Image-2 improves consistency and legibility for in-image text across infographics, diagrams, and slides — reducing the gap between prompt and output. Try it for yourself.
0:30

Text is often the hardest part of image generation to get right. MAI-Image-2 improves consistency and legibility for in-image text across infographics, diagrams, and slides — reducing the gap between prompt and output. Try it for yourself.

Microsoft AI

30,849 Aufrufe • vor 4 Monaten

Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution paper page: Text-based diffusion models have exhibited remarkable success in generation and editing, showing great promise for enhancing visual content with their generative prior. However, applying these models to video super-resolution remains challenging due to the high demands for output fidelity and temporal consistency, which is complicated by the inherent randomness in diffusion models. Our study introduces Upscale-A-Video, a text-guided latent diffusion framework for video upscaling. This framework ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences; globally, without training, a flow-guided recurrent latent propagation module is introduced to enhance overall video stability by propagating and fusing latent across the entire sequences. Thanks to the diffusion paradigm, our model also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation, enabling a trade-off between fidelity and quality. Extensive experiments show that Upscale-A-Video surpasses existing methods in both synthetic and real-world benchmarks, as well as in AI-generated videos, showcasing impressive visual realism and temporal consistency.
1:45

Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution paper page: Text-based diffusion models have exhibited remarkable success in generation and editing, showing great promise for enhancing visual content with their generative prior. However, applying these models to video super-resolution remains challenging due to the high demands for output fidelity and temporal consistency, which is complicated by the inherent randomness in diffusion models. Our study introduces Upscale-A-Video, a text-guided latent diffusion framework for video upscaling. This framework ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences; globally, without training, a flow-guided recurrent latent propagation module is introduced to enhance overall video stability by propagating and fusing latent across the entire sequences. Thanks to the diffusion paradigm, our model also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation, enabling a trade-off between fidelity and quality. Extensive experiments show that Upscale-A-Video surpasses existing methods in both synthetic and real-world benchmarks, as well as in AI-generated videos, showcasing impressive visual realism and temporal consistency.

AK

32,849 Aufrufe • vor 2 Jahren

ByteDance just announced MagicVideo-V2 Multi-Stage High-Aesthetic Video Generation paper page: The growing demand for high-fidelity video generation from textual descriptions has catalyzed significant research in this field. In this work, we introduce MagicVideo-V2 that integrates the text-to-image model, video motion generator, reference image embedding module and frame interpolation module into an end-to-end video generation pipeline. Benefiting from these architecture designs, MagicVideo-V2 can generate an aesthetically pleasing, high-resolution video with remarkable fidelity and smoothness. It demonstrates superior performance over leading Text-to-Video systems such as Runway, Pika 1.0, Morph, Moon Valley and Stable Video Diffusion model via user evaluation at large scale
2:07

ByteDance just announced MagicVideo-V2 Multi-Stage High-Aesthetic Video Generation paper page: The growing demand for high-fidelity video generation from textual descriptions has catalyzed significant research in this field. In this work, we introduce MagicVideo-V2 that integrates the text-to-image model, video motion generator, reference image embedding module and frame interpolation module into an end-to-end video generation pipeline. Benefiting from these architecture designs, MagicVideo-V2 can generate an aesthetically pleasing, high-resolution video with remarkable fidelity and smoothness. It demonstrates superior performance over leading Text-to-Video systems such as Runway, Pika 1.0, Morph, Moon Valley and Stable Video Diffusion model via user evaluation at large scale

AK

181,036 Aufrufe • vor 2 Jahren

Go from weeks of visual effects work to instant visualization with Gen-4.5 Image to Video. Just drop an image in and ask for what you want. Looking for more tips? Check out our full collection of tutorial and guides at the link below.
0:35

Go from weeks of visual effects work to instant visualization with Gen-4.5 Image to Video. Just drop an image in and ask for what you want. Looking for more tips? Check out our full collection of tutorial and guides at the link below.

Runway

10,586 Aufrufe • vor 7 Monaten

Wan2.5: Let Sound Take the Director’s Chair! 🎬 Today, we’re excited to unveil another major feature in our powerful Wan 2.5 Preview: Native Audio-Driven Video Generation. ✨ Now you can use audio input directly for both text-to-video and image-to-video generation. Combine audio with text prompts or a reference image to shape your video's narrative. ✨ With support for videos up to 10 seconds and enhanced video quality, unlock a richer visual space where more engaging stories come to life.
0:33

Wan2.5: Let Sound Take the Director’s Chair! 🎬 Today, we’re excited to unveil another major feature in our powerful Wan 2.5 Preview: Native Audio-Driven Video Generation. ✨ Now you can use audio input directly for both text-to-video and image-to-video generation. Combine audio with text prompts or a reference image to shape your video's narrative. ✨ With support for videos up to 10 seconds and enhanced video quality, unlock a richer visual space where more engaging stories come to life.

Wan

52,172 Aufrufe • vor 11 Monaten

Higgsfield Mod for Minecraft is live. > prompt any building or city, even the Statue of Liberty > create paintings with text-to-image > snap a view and restyle it with image-to-image > make videos from a prompt with text-to-video. > animate in-game photos with image-to-video
0:34

Higgsfield Mod for Minecraft is live. > prompt any building or city, even the Statue of Liberty > create paintings with text-to-image > snap a view and restyle it with image-to-image > make videos from a prompt with text-to-video. > animate in-game photos with image-to-video

Higgsfield AI 🧩

462,073 Aufrufe • vor 2 Monaten