This is Hollywood grade lip-syncing: Here is how to create an AI avatar with accurate and realistic Lip-Sync. This might be currently the best tool for video-to-video right now. Only 4 steps are needed:

1. Generate an image in @midjourney. We will use this image as a prompt for our video model. 2. Upload the images to @Kling_ai and create a video of your character talking. Important: the character needs to be talking.

3. Upload the video from Kling AI to @lipdubai. 4. Create your voiceover in @elevenlabsio and upload the audio file to LipDub AI as well. Now you just wait and LipDub AI will generate a video with the lip sync aligned with the audio you uploaded from ElevenLabs. That's it!

This post is NOT sponsored, like almost all of my posts. I'm just sharing this workflow, because I think it brings value. LipDub AI is currently in beta, so you might have to reach out to them. If you want to share the knowledge with others, feel free to repost the post below ❤️

If you’re interested in more workflows and tutorials like these, check my free AI-Tutorial section here:

The lip-syncing is better than most, but still not there. I think it's going to take a few years before they nail this down. Lip/tongue/cheek movements during speech are super nuanced. Even the slightest deviation in those dynamics is instantly noticeable & jarring to most humans

Thank you, Vic. I agree with you that it's not perfect, but I also think you give lip sync way too much weight. It just needs to be somewhat accurate. The mimicry and facial expressions are way more important and I think that this what will make the difference.

Very cool! Does it work with other face angles?

Looks great

Thank you, Chrissie :)

Thanks! Looks good! How do you think it compares to Heygen?

HeyGen does not allow training AI avatars, only real humans, but the quality is almost similar. On the other hand, LipDub AI takes way more time to train a model than HeyGen.