Tests using a #metahuman from #UnrealEngine5 with #stablediffusion Multi-ControlNet. Applied diff. amounts of deflickering in #DaVinciResolve. Mainly seeing how it handles 3D camera moves. #aicinema #controlnet #aiia Unreal Engine #aiphotography #MachineLearning #DeepLearning

@UnrealEngine I've been wanting to do something like this with my cc4 characters. The way you're doing it, it really takes the edge off

@Lynncorrigible @UnrealEngine I'm gonna look into iClone soon! Really like their latest update with dynamic skin wrinkles driven by expressions!

@UnrealEngine Here’s one of our interactive avatars built last year using UE4 and ML auto-encoders for appearance and Nvidia’s Nemo for the voice: UE5 versions coming soon.

@UnrealEngine Duuuuuuude!! I'm brimming!! This is exactly what I planned on doing once I order my new graphics card (on a GTX 970 don't laugh 😅) Bravo! This mixed with a mo-cap suit and/or when Gen-1 evolves, making entire pilots for movies now totally possible. God it gets me pumped

@UnrealEngine Yes! Really going to be interesting seeing the new work flows that ppl are going to innovate! Especially as the tech continues to improve!

The sticking textures reminds me a lot of the phenomenon observed in StyleGANv2. I don't know enough about the innards of ControlNet to say, but maybe there's a similar underlying cause?

Demonstrating a technique to further reduce flickering - instead of using each frame, taking every 5th frame and then interpolating between the gap using Google's FILM interpolation - I think if you ran this on the initial post Controlnet video and then used DaVinci resolve it'd be close to perfect (this is done using an open-source tool I'm building)

@UnrealEngine Fantastic!

@danberridge @UnrealEngine 🙌

@UnrealEngine This is the way ;)

@UnrealEngine 🙌