Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Collaborative Video Diffusion Consistent Multi-video Generation with Camera Control Research on video generation has recently made tremendous progress, enabling high-quality videos to be generated from text prompts or images. Adding control to the video generation

_akhaliq's profile picture

AK

502,269 subscribers

29,262 Aufrufe • vor 2 Jahren •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

6 Kommentare

Profilbild von AK
AKvor 2 Jahren

process is an important goal moving forward and recent approaches that condition video generation models on camera trajectories make strides towards it. Yet, it remains challenging to generate a video of the same scene from multiple different camera trajectories.

Profilbild von AK
AKvor 2 Jahren

Solutions to this multi-video generation problem could enable large-scale 3D scene generation with editable camera trajectories, among other applications. We introduce collaborative video diffusion (CVD) as an important step towards this vision. The CVD framework includes a

Profilbild von AK
AKvor 2 Jahren

novel cross-video synchronization module that promotes consistency between corresponding frames of the same video rendered from different camera poses using an epipolar attention mechanism. Trained on top of a state-of-the-art camera-control module for video generation,

Profilbild von AK
AKvor 2 Jahren

CVD generates multiple videos rendered from different camera trajectories with significantly better consistency than baselines, as shown in extensive experiments.

Profilbild von AK
AKvor 2 Jahren

paper page:

Profilbild von AK
AKvor 2 Jahren

daily papers:

Ähnliche Videos

BlockVid Block Diffusion for High-Quality and Consistent Minute-Long Video Generation
1:00

BlockVid Block Diffusion for High-Quality and Consistent Minute-Long Video Generation

AK

19,864 Aufrufe • vor 6 Monaten

Nvidia just dropped GEN3C 3D-Informed World-Consistent Video Generation with Precise Camera Control
1:18

Nvidia just dropped GEN3C 3D-Informed World-Consistent Video Generation with Precise Camera Control

AK

58,146 Aufrufe • vor 1 Jahr

V3D Video Diffusion Models are Effective 3D Generators Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency
0:21

V3D Video Diffusion Models are Effective 3D Generators Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency

AK

31,997 Aufrufe • vor 2 Jahren

World meet #emuvideo For the past year, our team has been pushing on video generation. The result? Emu Video that generates high quality videos from text or images. SOTA performance vs. commercial products and academic papers. Check it out
1:29

World meet #emuvideo For the past year, our team has been pushing on video generation. The result? Emu Video that generates high quality videos from text or images. SOTA performance vs. commercial products and academic papers. Check it out

Ishan Misra

132,048 Aufrufe • vor 2 Jahren

Grok Imagine API just released A world-class video generation + video editing model Text-to-Video: Turn simple prompts into rich video clips with audio Image Generation + Editing: Bring ideas to life with visuals from scratch Video Editing Tools: Restyle scenes, add/remove props, control motion Best-in-Class Quality + Low Latency: Designed to deliver fast, cost-efficient results API pricing: Image input: $0.002 Video input : $0.01 Video output : $0.05
0:22

Grok Imagine API just released A world-class video generation + video editing model Text-to-Video: Turn simple prompts into rich video clips with audio Image Generation + Editing: Bring ideas to life with visuals from scratch Video Editing Tools: Restyle scenes, add/remove props, control motion Best-in-Class Quality + Low Latency: Designed to deliver fast, cost-efficient results API pricing: Image input: $0.002 Video input : $0.01 Video output : $0.05

X Freeze

15,078 Aufrufe • vor 4 Monaten

(1/2) LightIt: Illumination Modeling and Control for Diffusion Models! #CVPR2024 We facilitate lighting control for novel image generation from text prompts. We can also edit lighting for a given input image. Video: Project:
0:31

(1/2) LightIt: Illumination Modeling and Control for Diffusion Models! #CVPR2024 We facilitate lighting control for novel image generation from text prompts. We can also edit lighting for a given input image. Video: Project:

Matthias Niessner

19,781 Aufrufe • vor 2 Jahren

SynCamMaster Synchronizing Multi-Camera Video Generation from Diverse Viewpoints
0:31

SynCamMaster Synchronizing Multi-Camera Video Generation from Diverse Viewpoints

AK

20,880 Aufrufe • vor 1 Jahr

Make Pixels Dance: High-Dynamic Video Generation paper page: Creating high-dynamic videos such as motion-rich actions and sophisticated visual effects poses a significant challenge in the field of artificial intelligence. Unfortunately, current state-of-the-art video generation methods, primarily focusing on text-to-video generation, tend to produce video clips with minimal motions despite maintaining high fidelity. We argue that relying solely on text instructions is insufficient and suboptimal for video generation. In this paper, we introduce PixelDance, a novel approach based on diffusion models that incorporates image instructions for both the first and last frames in conjunction with text instructions for video generation. Comprehensive experimental results demonstrate that PixelDance trained with public data exhibits significantly better proficiency in synthesizing videos with complex scenes and intricate motions, setting a new standard for video generation.
3:09

Make Pixels Dance: High-Dynamic Video Generation paper page: Creating high-dynamic videos such as motion-rich actions and sophisticated visual effects poses a significant challenge in the field of artificial intelligence. Unfortunately, current state-of-the-art video generation methods, primarily focusing on text-to-video generation, tend to produce video clips with minimal motions despite maintaining high fidelity. We argue that relying solely on text instructions is insufficient and suboptimal for video generation. In this paper, we introduce PixelDance, a novel approach based on diffusion models that incorporates image instructions for both the first and last frames in conjunction with text instructions for video generation. Comprehensive experimental results demonstrate that PixelDance trained with public data exhibits significantly better proficiency in synthesizing videos with complex scenes and intricate motions, setting a new standard for video generation.

AK

101,514 Aufrufe • vor 2 Jahren

High quality AI generated talking heads are coming! GAIA can generate talking avatars from a single portrait image and speech clip. It even supports text prompts like `sad`, `open mouth` or `surprise` to guide video generation. Crazy times ahead 🤯
0:26

High quality AI generated talking heads are coming! GAIA can generate talking avatars from a single portrait image and speech clip. It even supports text prompts like `sad`, `open mouth` or `surprise` to guide video generation. Crazy times ahead 🤯

Dreaming Tulpa 🥓👑

659,956 Aufrufe • vor 2 Jahren

Today we’re sharing two new advances in our generative AI research: Emu Video & Emu Edit. Details ➡️ These new models deliver exciting results in high quality, diffusion-based text-to-video generation & controlled image editing w/ text instructions. 🧵
0:33

Today we’re sharing two new advances in our generative AI research: Emu Video & Emu Edit. Details ➡️ These new models deliver exciting results in high quality, diffusion-based text-to-video generation & controlled image editing w/ text instructions. 🧵

AI at Meta

798,096 Aufrufe • vor 2 Jahren

MuseSteamer just leveled up! Our video generation model now supports real-time interactive long-form video generation. It breaks the traditional 10-second limit, creating videos of any length with greater speed and control—enabling users to pause, rewrite storylines, or extend transitions any time during the process.
0:22

MuseSteamer just leveled up! Our video generation model now supports real-time interactive long-form video generation. It breaks the traditional 10-second limit, creating videos of any length with greater speed and control—enabling users to pause, rewrite storylines, or extend transitions any time during the process.

Baidu Inc.

142,732 Aufrufe • vor 7 Monaten

BulletTime Decoupled Control of Time and Camera Pose for Video Generation
1:36

BulletTime Decoupled Control of Time and Camera Pose for Video Generation

AK

11,745 Aufrufe • vor 6 Monaten

AI video generation finally becomes directable. Use real actor motion to drive your characters — and control the camera for cinematic shots. Consistent motion. Consistent framing. Real creative power.
0:30

AI video generation finally becomes directable. Use real actor motion to drive your characters — and control the camera for cinematic shots. Consistent motion. Consistent framing. Real creative power.

Kinetix

771,932 Aufrufe • vor 6 Monaten

Video generation is now live on OpenRouter! One API gives you access to the top video models, right alongside your text, images, audio, embeddings, and reranker generations. Watch our human-generated launch video:
3:27

Video generation is now live on OpenRouter! One API gives you access to the top video models, right alongside your text, images, audio, embeddings, and reranker generations. Watch our human-generated launch video:

OpenRouter

119,408 Aufrufe • vor 1 Monat

Self-Forcing++ for minute-scale video generation ByteDance's new method generates high-quality videos up to 4 min 15 sec! It scales diffusion models without long-video teachers or retraining, preserving fidelity and consistency.
1:40

Self-Forcing++ for minute-scale video generation ByteDance's new method generates high-quality videos up to 4 min 15 sec! It scales diffusion models without long-video teachers or retraining, preserving fidelity and consistency.

DailyPapers

17,720 Aufrufe • vor 8 Monaten

🚀 Long shot generation in LTXV-13B Now supports up to 60 seconds of video! • Auto-regressive generation (up to 60s) • Standard generation (up to 20s) • Streamable on H100 with low latency • Compatible with control LoRAs (released last week) • Time-varying prompts supported
3:30

🚀 Long shot generation in LTXV-13B Now supports up to 60 seconds of video! • Auto-regressive generation (up to 60s) • Standard generation (up to 20s) • Streamable on H100 with low latency • Compatible with control LoRAs (released last week) • Time-varying prompts supported

Yoav HaCohen

13,848 Aufrufe • vor 10 Monaten

StreamDiT Real-Time Streaming Text-to-Video Generation StreamDiT enables real-time text-to-video generation at 16 FPS on a single GPU (H100)
1:00

StreamDiT Real-Time Streaming Text-to-Video Generation StreamDiT enables real-time text-to-video generation at 16 FPS on a single GPU (H100)

AK

43,659 Aufrufe • vor 11 Monaten

Introducing varg teams Collaborative mode for video generation. ✦ Create videos with AI agents ✦ Share assets and prompts ✦ Fork and remix any video Every video in varg is code - full end-to-end generated videos Built for teams making ads, social media, and in-app content
1:07

Introducing varg teams Collaborative mode for video generation. ✦ Create videos with AI agents ✦ Share assets and prompts ✦ Fork and remix any video Every video in varg is code - full end-to-end generated videos Built for teams making ads, social media, and in-app content

Alex Varga /✦

23,768 Aufrufe • vor 7 Tagen

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:
0:46

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:

Yuan Liu

33,363 Aufrufe • vor 1 Jahr