Jag the Hero
@JagTheHero • 3,493 subscribers
32M | Software Engineer | America First 🇺🇸 MAGA | https://t.co/ONigVklruq The Guild Discord - Games, Anime, AI Art. No wokies!
Videos
Today I made a prototype of a system that turns a single AI-generated portrait into a living VTuber. No rigging, no Live2D, nobody spending 80 hours in an editor. The character was generated locally with SDXL using the same model and custom LoRA I use for the art bot in my Discord server. A video model (Grok) then animated her into a chain of idle clips, each one starting on the exact frame where the previous one ended, so she loops forever and rarely cuts. Her mouth is the interesting part: a tracker follows the drawn lines of her lips through every frame, measuring position, angle, and width, and a composited mouth rides along, rotating and foreshortening as her head moves. The mouth shapes themselves were inpainted onto her own face by SDXL, so every viseme is in her exact art style. Real phoneme detection drives which shape shows. Because the mouth is composited rather than baked in, it stops the instant the voice stops. The body animation never even knows speech exists. Animating her ate about half of my SuperGrok credits, which works out to $14.32 if you paid for the same experimentation directly. I used 5 of the 12 generated clips in this demo. Narrated this demo using the Kokoro TTS model for free. Next step would be webcam emotion detection so she can swap between animation sets when the streamer smiles or gasps, more mouth shapes
Jag the Hero273,586 просмотров • 13 дней назад
Больше нет контента для загрузки