Loading video...

Video Failed to Load

Go Home

InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbing

PurzBeats's profile picture

Purz.ai

16,701 subscribers

16,711 views • 1 year ago •via X (Twitter)

Science & TechnologyEducationArts
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Radial Attention O(nlog n) Sparse Attention with Energy Decay for Long Video Generation
1:03

Radial Attention O(nlog n) Sparse Attention with Energy Decay for Long Video Generation

AK

36,271 views • 1 year ago

FramePack is out Packing Input Frame Context in Next-Frame Prediction Models for Video Generation
0:56

FramePack is out Packing Input Frame Context in Next-Frame Prediction Models for Video Generation

AK

31,486 views • 1 year ago

More examples of what Meta Movie Gen can do across video generation, precise video editing, personalized video generation and audio generation.
1:08

More examples of what Meta Movie Gen can do across video generation, precise video editing, personalized video generation and audio generation.

AI at Meta

97,085 views • 1 year ago

🔥 Just launched: InfiniteTalk is now LIVE on WaveSpeedAI! Create endless conversations with ultra-realistic voices — now supporting up to 2 minutes of video generation! 🚀🎬 Try it now 👉 #WaveSpeedAI #InfiniteTalk #AIvoice #AIvideo #GenerativeAI #AIfuture
0:25

🔥 Just launched: InfiniteTalk is now LIVE on WaveSpeedAI! Create endless conversations with ultra-realistic voices — now supporting up to 2 minutes of video generation! 🚀🎬 Try it now 👉 #WaveSpeedAI #InfiniteTalk #AIvoice #AIvideo #GenerativeAI #AIfuture

WaveSpeedAI

16,052 views • 11 months ago

LTX-2 is natively supported in ComfyUI on Day 0 🎬🔊 The next chapter in controllability for open-source video generation. - Open-source audio-video foundation model - Generates motion, dialogue, SFX, and music together - Canny, Depth & Pose video-to-video control - Keyframe-driven generation - Native upscaling and prompt enhancement Everything in a single pass.
0:36

LTX-2 is natively supported in ComfyUI on Day 0 🎬🔊 The next chapter in controllability for open-source video generation. - Open-source audio-video foundation model - Generates motion, dialogue, SFX, and music together - Canny, Depth & Pose video-to-video control - Keyframe-driven generation - Native upscaling and prompt enhancement Everything in a single pass.

ComfyUI

108,333 views • 7 months ago

Meet LongCat-Video-Avatar: a robust audio-driven avatar model that pushes the boundaries of long-form video generation. Compared with the previous InfiniteTalk, LongCat-Video-Avatar delivers far better long-sequence stability and realism. New highlights： ⚙ Built on the LongCat-Video architecture, now supporting Audio-Text-to-Video (AT2V), Audio-Text-Image-to-Video (ATI2V), and Video Continuation modes. 🎭 Open-source SOTA Realism: Ranked 1st in overall anthropomorphism scores for both single and multi-subject scenarios in EvalTalker evaluations (492 participants, 3 independent raters per video). ♾ High-quality long videos: Cross-Chunk Latent Stitching prevents pixel degradation and error accumulation over time, ensuring seamless stitching quality. 🔒 Long-term consistency: Reference Skip Attention maintains ID consistency while eliminating rigid copy-paste artifacts. 🪄 Supports multi-person and infinite-length video generation. 🔗Open-sourced Code: Hugging Face: Project: Paper:
5:00

Meet LongCat-Video-Avatar: a robust audio-driven avatar model that pushes the boundaries of long-form video generation. Compared with the previous InfiniteTalk, LongCat-Video-Avatar delivers far better long-sequence stability and realism. New highlights： ⚙ Built on the LongCat-Video architecture, now supporting Audio-Text-to-Video (AT2V), Audio-Text-Image-to-Video (ATI2V), and Video Continuation modes. 🎭 Open-source SOTA Realism: Ranked 1st in overall anthropomorphism scores for both single and multi-subject scenarios in EvalTalker evaluations (492 participants, 3 independent raters per video). ♾ High-quality long videos: Cross-Chunk Latent Stitching prevents pixel degradation and error accumulation over time, ensuring seamless stitching quality. 🔒 Long-term consistency: Reference Skip Attention maintains ID consistency while eliminating rigid copy-paste artifacts. 🪄 Supports multi-person and infinite-length video generation. 🔗Open-sourced Code: Hugging Face: Project: Paper:

Meituan LongCat

28,170 views • 8 months ago

Wan2.5: Let Sound Take the Director’s Chair! 🎬 Today, we’re excited to unveil another major feature in our powerful Wan 2.5 Preview: Native Audio-Driven Video Generation. ✨ Now you can use audio input directly for both text-to-video and image-to-video generation. Combine audio with text prompts or a reference image to shape your video's narrative. ✨ With support for videos up to 10 seconds and enhanced video quality, unlock a richer visual space where more engaging stories come to life.
0:33

Wan2.5: Let Sound Take the Director’s Chair! 🎬 Today, we’re excited to unveil another major feature in our powerful Wan 2.5 Preview: Native Audio-Driven Video Generation. ✨ Now you can use audio input directly for both text-to-video and image-to-video generation. Combine audio with text prompts or a reference image to shape your video's narrative. ✨ With support for videos up to 10 seconds and enhanced video quality, unlock a richer visual space where more engaging stories come to life.

Wan

52,172 views • 10 months ago

Live Avatar Streaming Real-time Audio-Driven Avatar Generation with Infinite Length
18:07

Live Avatar Streaming Real-time Audio-Driven Avatar Generation with Infinite Length

AK

23,009 views • 8 months ago

New open-source LoRA built in-house: LTX2.3 Audio Reactive LoRA. Built for music-driven video generation: stronger beat sync, graphic texture, color separation, light pulses, particles, and shapes that move with the track. Model:
0:39

New open-source LoRA built in-house: LTX2.3 Audio Reactive LoRA. Built for music-driven video generation: stronger beat sync, graphic texture, color separation, light pulses, particles, and shapes that move with the track. Model:

fal

16,472 views • 2 months ago

Ovi is out on Hugging Face Twin Backbone Cross-Modal Fusion for Audio-Video Generation Ovi is a veo-3 like, video+audio generation model that simultaneously generates both video and audio content from text or text+image inputs. Video+Audio Generation: Generate synchronized video and audio content simultaneously Flexible Input: Supports text-only or text+image conditioning 5-second Videos: Generates 5-second videos at 24 FPS, area of 720×720, at various aspect ratios (9:16, 16:9, 1:1, etc)
2:08

Ovi is out on Hugging Face Twin Backbone Cross-Modal Fusion for Audio-Video Generation Ovi is a veo-3 like, video+audio generation model that simultaneously generates both video and audio content from text or text+image inputs. Video+Audio Generation: Generate synchronized video and audio content simultaneously Flexible Input: Supports text-only or text+image conditioning 5-second Videos: Generates 5-second videos at 24 FPS, area of 720×720, at various aspect ratios (9:16, 16:9, 1:1, etc)

AK

23,082 views • 10 months ago

Video without dubbing
0:23

Video without dubbing

Serhii Sternenko

15,508 views • 1 year ago

#Sarvam AI's video dubbing is insanely amazing. The most interesting part is that while dubbing, it doesn't use a random voice — it clones the original voice instead. It supports dubbing for multiple people in a single frame, and that too for clips up to 3 minutes long. I just tried a small clip, and the results are amazing!
0:41

#Sarvam AI's video dubbing is insanely amazing. The most interesting part is that while dubbing, it doesn't use a random voice — it clones the original voice instead. It supports dubbing for multiple people in a single frame, and that too for clips up to 3 minutes long. I just tried a small clip, and the results are amazing!

Chandrashekhar Dhage

62,832 views • 6 months ago

Google presents VLOGGER Multimodal Diffusion for Embodied Avatar Synthesis We propose VLOGGER, a method for audio-driven human video generation from a single input image of a person, which builds on the success of recent generative diffusion models. Our method consists of
0:22

Google presents VLOGGER Multimodal Diffusion for Embodied Avatar Synthesis We propose VLOGGER, a method for audio-driven human video generation from a single input image of a person, which builds on the success of recent generative diffusion models. Our method consists of

AK

66,375 views • 2 years ago

Controllability matters for video generation. We thus propose SparseCtrl that enables keyframe (sketch, depth, img) animation/transition/prediction/interpolation by feeding TEMPORALLY SPARSE condition maps. It’s also compatible with #AnimateDiff. Page:
2:19

Controllability matters for video generation. We thus propose SparseCtrl that enables keyframe (sketch, depth, img) animation/transition/prediction/interpolation by feeding TEMPORALLY SPARSE condition maps. It’s also compatible with #AnimateDiff. Page:

Ceyuan Yang

39,542 views • 2 years ago

Original video without dubbing.
0:32

Original video without dubbing.

Serhii Sternenko

13,249 views • 1 year ago

yet ANOTHER release from ByteDance just landed on the hub ✨HuMo✨ > video generation w/ multi-modal conditioning: audio, text & image > supports consistent subject preservation, synchronized audio-driven motion > based on Wan 2.1 & Whisper Large v3
1:51

yet ANOTHER release from ByteDance just landed on the hub ✨HuMo✨ > video generation w/ multi-modal conditioning: audio, text & image > supports consistent subject preservation, synchronized audio-driven motion > based on Wan 2.1 & Whisper Large v3

Linoy Tsaban🎗️

17,573 views • 11 months ago

Can we simplify video generation by decomposing it into interleaved text-video co-generation? Would explicit, repeated thinking in language improve generation in pixels? We introduce TV2TV: a unified model that jointly learns - language modeling (next-token prediction) - video flow matching (next-frame prediction) At inference, TV2TV dynamically alternates between textual thinking and video generation. Model generations below: interleaved text plans and video slices (~1–2s) are co-generated over time, conditioned on a single frame per sport. 📖
0:36

Can we simplify video generation by decomposing it into interleaved text-video co-generation? Would explicit, repeated thinking in language improve generation in pixels? We introduce TV2TV: a unified model that jointly learns - language modeling (next-token prediction) - video flow matching (next-frame prediction) At inference, TV2TV dynamically alternates between textual thinking and video generation. Model generations below: interleaved text plans and video slices (~1–2s) are co-generated over time, conditioned on a single frame per sport. 📖

Xiaochuang Han

32,066 views • 8 months ago

Waiting minutes for AI video generation is over. Runway's new model on NVIDIA Vera Rubin hits under 100ms to first frame. Minds are blown.
0:53

Waiting minutes for AI video generation is over. Runway's new model on NVIDIA Vera Rubin hits under 100ms to first frame. Minds are blown.

Min Choi

385,217 views • 5 months ago

Glad to share Seaweed-7B, a cost-effective foundation model for video generation. Our tech report highlights the key designs that significantly improve compute efficiency and performance given limited resources, achieving comparable quality against other industry-level models. To unleash the power of the foundation model, Seaweed-7B further enables a wide range of downstream applications including image-to-video generation, human video generation, subject-consistent video generation, video-audio joint generation, long video generation and storytelling, real-time generation, super-resolution generation, camera controlled generation. Check out our webpage and report for more details: Webpage: Paper: It's a wonderful journey of the last year. Thanks to all teammates for their contributions, sincerely.
1:30

Glad to share Seaweed-7B, a cost-effective foundation model for video generation. Our tech report highlights the key designs that significantly improve compute efficiency and performance given limited resources, achieving comparable quality against other industry-level models. To unleash the power of the foundation model, Seaweed-7B further enables a wide range of downstream applications including image-to-video generation, human video generation, subject-consistent video generation, video-audio joint generation, long video generation and storytelling, real-time generation, super-resolution generation, camera controlled generation. Check out our webpage and report for more details: Webpage: Paper: It's a wonderful journey of the last year. Thanks to all teammates for their contributions, sincerely.

Ceyuan Yang

77,502 views • 1 year ago

Introducing Imagine v0.9, our new video generation model with massive upgrades from v0.1 in visual quality, motion, audio generation, and more. Now available for free on all our products:
0:25

Introducing Imagine v0.9, our new video generation model with massive upgrades from v0.1 in visual quality, motion, audio generation, and more. Now available for free on all our products:

SpaceXAI

8,314,608 views • 10 months ago