we just shipped our most capable video-to-video lipsyncing model yet introducing lipsync-1.8.0 🪄 edit what anyone says in any video, no training required available now via API + v2 playground. check it out 👇

sign up and get $5 in free credits to try it out building intuitive experiences around custom models isn't trivial v1 was good, but not extensible to new workflows + model capabilities so, we redesigned + rebuilt it from the ground up :) (2/6)

lipsync-1.8.0 marks a fundamental shift in our model architecture we gained the ability to adapt to an identity of a target speaker w/o training at runtime this lets us generate fast + accurate results unique to every person ex. we generated a missing tooth after lipsync 👇 (3/6)

prev. models + OSS (incl. wav2lip) struggle to generate/edit highly expressive speech it's partly bc they take input lip shapes as important information when generating outputs 1.8.0 has learned to generate the lip shape irrespective of the input lip shapes ^this gives us more freedom in controlling expressiveness of the output + works with big smiles (4/6)

we benchmarked against the most popular OSS models, including our own (wav2lip) we consistently outperform in visual quality, accuracy, and identity preservation, and expressive lipsync (5/6)

to generate video, you must first learn a foundational understanding of it – and that is precisely what is hard we've constrained the problem to human motion, specifically how human faces react / interact w/ speech ^doing this, we've learned a foundational understanding of a human face. this is exciting! it means lipsync is the first task we've solved for, but we can tackle other fine-grained editing tasks related to the human face by playing in the same latent space looking forward to showing you what comes next :) (6/6)

Congrats 🔥 Nice cameo @rbhuta95

@rbhuta95 🙏🏼 haha come thru more often

Bros are cooking

always 👌

shipping from the mountains 🚀🚀🚀