Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

We just tested a lip sync workflow in Seedance 2.0… and it might actually compete with HeyGen and other avatar models 👀 You upload: – a reference image – a blacked out video with audio OR audio file – a prompt (camera motion + VO) The prompted VO is...

19,265 görüntüleme • 3 ay önce •via X (Twitter)

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Seedance 2.0 is allowing us to enter a new era of music video creation. Here is how I created HONEY. It was a quick test to see how well this workflow holds up. 🐝 1 - Write your song and generate the music with Suno 5.5. 2 - Use an image generator of your choice. For HONEY I combined both Grok Imagine for aesthetics and Nano Banana Pro for refined editing. 3 - In Capcut I import my audio and just save out a blank video video containing the audio. This step is important because this video file containing audio will now be used with Seedance 2.0 as a video reference with Omni. This allows the AI to apply automatic and realistic lipsync and movement to the music, it's extremely powerful! 4 - Once I have a both my image and video with audio as reference, I use Seedance 2.0 Omni and upload my starting image and then the video reference with the audio. 5 - From here I'm simply prompting like normal, specifying what's happening in my scene with detailed instructions, mentioning multi shots and camera angle changes and then specifying that the person is singing along to the song. I type out the lyrics that are present to have better lipsync accuracy. 6 - Once I have generated a video and like the result, I do video to video, so i upload that video that just got generated and type "The scene continues" and prompt new actions to take place. This allows you to expand on a narrative. These new shots can be used as B-ROLL and since I uploaded my video as reference I have full consistency of everything it saw in the video. This is also extremely powerful. 7 - This is actually the most difficult part. Edit in Capcut. This is where you need to understand pacing and shot selection from all the scenes you generated to bring it all together. You must be strategic with the editing. Goodluck! I'll probably record a video tutorial at some point as it's easier to see what is being done.

Travis Davids

19,236 görüntüleme • 3 ay önce

🇨🇳 Another great Chinese Model, OmniHuman-1.5 from ByteDance Turns 1 image plus a voice track into expressive avatar video by pairing a System 1 and System 2 inspired planner with a Diffusion Transformer, Produces coherent motion for over 1 minute with moving camera and multi character scenes. Most avatar models move to the beat of the audio but miss meaning, so gestures feel generic and emotions feel shallow. The fix here is a Multimodal LLM planner that listens to the speech and drafts a structured plan describing intent, emotions, beats, and high level actions, which gives the motion engine clear semantic targets instead of only rhythm. The motion engine is a Multimodal Diffusion Transformer that fuses the plan with audio, the single reference image, and optional text prompts, then synthesizes continuous body, face, and head motion that matches both words and tone. A key trick is a Pseudo Last Frame, a synthetic target that summarizes the next expected state, which stabilizes fusion across modalities and keeps motion consistent over long spans. From just 1 image and speech, the system outputs speaking avatars with synchronized lips, context aware gestures, and continuous camera movement, and it also supports multi character interactions without manual choreography. Reported results show strong lip sync accuracy, high video quality, natural motion, and close match to text prompts, and the same setup works on nonhuman characters too.

Rohan Paul

63,859 görüntüleme • 11 ay önce