Skip navigation
Meta just announced FlowVid
0:00 / 0:47

Meta just announced FlowVid

AK

@_akhaliq539K followers

Visit on X
687 likesDownloadOriginal post

123,811 views

Meta just announced FlowVid Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis paper page: Diffusion models have transformed the image-to-image (I2I) synthesis and are now permeating into videos. However, the advancement of video-to-video (V2V) synthesis has been hampered by the challenge of maintaining temporal consistency across video frames. This paper proposes a consistent V2V synthesis framework by jointly leveraging spatial conditions and temporal optical flow clues within the source video. Contrary to prior methods that strictly adhere to optical flow, our approach harnesses its benefits while handling the imperfection in flow estimation. We encode the optical flow via warping from the first frame and serve it as a supplementary reference in the diffusion model. This enables our model for video synthesis by editing the first frame with any prevalent I2I models and then propagating edits to successive frames. Our V2V model, FlowVid, demonstrates remarkable properties: (1) Flexibility: FlowVid works seamlessly with existing I2I models, facilitating various modifications, including stylization, object swaps, and local edits. (2) Efficiency: Generation of a 4-second video with 30 FPS and 512x512 resolution takes only 1.5 minutes, which is 3.1x, 7.2x, and 10.5x faster than CoDeF, Rerender, and TokenFlow, respectively. (3) High-quality: In user studies, our FlowVid is preferred 45.7% of the time, outperforming CoDeF (3.5%), Rerender (10.2%), and TokenFlow (40.4%).

9 Comments

Comments from the original post on X (Twitter)

  • Furkan Gözükara's profile picture

    @GozukaraFurkan

    I have rerender tutorial and auto installer

    5 likes

  • JonathanAlis's profile picture

    @jonathanalis1

    Wow, messy optical flow have been degrading one of my works. I think it would be helpful. Tkx por sharing. ^^

    2 likes

  • Sahar Mor's profile picture

    @theaievangelist

    Research tl;dr

    1 like

  • stopscammers's profile picture

    @stopscammers00

    Ahhh get over it

    1 like

  • kfant's profile picture

    @6___0

    the question is will they open source it?

  • Vadim Kozlov's profile picture

    @Vajayjay_K

    You know it's just going to keep getting better and better

  • Death Star Robot 🇺🇸 🇺🇦 🇮🇱 🇹🇼's profile picture

    @DeathStarRobot

    Style transfer looked better.

  • stopscammers's profile picture

    @stopscammers00

    Meta is zinosm platform who steal x platform

  • Mamadclark's profile picture

    @mamadclark

    boycott meta