📢 Introducing DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models Compared to vanilla DPO, we improve paired data construction and preference label granularity, leading to better visual quality and motion strength with only 1/3 of the data. 🧵

Project page: Paper: Work during my internship at @Snap with the amazing collaborators @anilkagak2, @isskoro, @WilliMenapace, @ashmrz10, @igilitschenski, @SergeyTulyakov, and @siarohin9013.

*Paired data construction* Traditional DPO generates video pairs from *independent noise maps*. Instead, DenseDPO samples videos from *partially noised GT videos*, leading to video pairs with similar global motion while allowing variations in local details.

*Dense temporal preference* Instead of using a single win/lose label for the entire video (e.g., 5s), we densely label each *short segment* (e.g., 1s). This fine-grained annotation prevents optimizing video frames with artifacts and greatly improves data efficiency.

*VLM dense labels* Prior works show that VLMs such as GPT struggle with long videos. We show that GPT is actually good at assessing short clips, achieving preference accuracy similar to task-specifically fine-tuned video reward models and DPO results close to using human labels.

Takeaways: - Videos are different from images due to the temporal axis; - We really need fine-grained supervision to improve video generation, e.g., the spatio-temporal motion repr in VideoJAM, the dense temporal captions in my previous work MinT, and the dense preferences here.