Loading video...

Video Failed to Load

Go Home

This workflow gave me best lip sync on a moving character. Workflow below.

68,857 views • 1 year ago •via X (Twitter)

12 Comments

madpencil_'s profile picture
madpencil_1 year ago

~ created the image, I used @freepik imagen 3 model for the most realistic outputs. ~ Animated the initial 5 sec clip using @LumaLabsAI Ray2 it's the best realistic body and camera motion currently.

madpencil_'s profile picture
madpencil_1 year ago

@LumaLabsAI ~ Then Use @PixVerse_ it is very underrated it has an incredible extend and lip sync feature, we just need to smartly decide which clip might work best. Using Pixverse extend your luma clip a bit longer by your length of Audio file to counter the pace of the video later.

madpencil_'s profile picture
madpencil_1 year ago

@LumaLabsAI @PixVerse_ If you find your extended parts to be a bit slower speed it up using editing tools to make it look like natural movement, I used Capcut.

madpencil_'s profile picture
madpencil_1 year ago

@LumaLabsAI @PixVerse_ Now again Upload your edited clip to Pixverse using the lip sync feature, it can lip sync up to 30 seconds. And you're done.

madpencil_'s profile picture
madpencil_1 year ago

For voice i used "Zonos" It has a voice cloning feature and inbuilt voice tamplates and different accents. It gives you 100 free minutes.

madpencil_'s profile picture
madpencil_1 year ago

Bonus Expand or experiment with styles on @runwayml , but wish we could get longer outputs, I love its incredible tools.

madpencil_'s profile picture
madpencil_1 year ago

Wish I had Topaz sub the end output might have looked freaking good 😅 Anyway if you find it interesting or useful give samsh the follow button 🙌

opensourceCM's profile picture
opensourceCM1 year ago

What’s the cost of mistakes in your contracts? If you work with contracts day-to-day, it’s time to automate. Track every detail, streamline workflows ... ✨ Make managing contracts as easy as a few clicks. Visit our new website & book your demo today!

Serge Doubinski's profile picture
Serge Doubinski1 year ago

Ok wow I thought that was lip sync on top of existing video, but the whole scene is synthetic? Very cool.

madpencil_'s profile picture
madpencil_1 year ago

Yes, as you can see it's from an image generated to the final result, thank you.

Royght's profile picture
Royght1 year ago

This is amazingly realistic!

madpencil_'s profile picture
madpencil_1 year ago

Turned out nice , thank you

Related Videos