Loading video...

Video Failed to Load

Go Home

📢 Introducing DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models Compared to vanilla DPO, we improve paired data construction and preference label granularity, leading to better visual quality and motion strength with only 1/3 of the data. 🧵

Dazitu_616's profile picture

Ziyi Wu

1,292 subscribers

35,608 views • 1 year ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

5 Comments

Ziyi Wu's profile picture
Ziyi Wu1 year ago

Project page: Paper: Work during my internship at @Snap with the amazing collaborators @anilkagak2, @isskoro, @WilliMenapace, @ashmrz10, @igilitschenski, @SergeyTulyakov, and @siarohin9013.

Ziyi Wu's profile picture
Ziyi Wu1 year ago

*Paired data construction* Traditional DPO generates video pairs from *independent noise maps*. Instead, DenseDPO samples videos from *partially noised GT videos*, leading to video pairs with similar global motion while allowing variations in local details.

Ziyi Wu's profile picture
Ziyi Wu1 year ago

*Dense temporal preference* Instead of using a single win/lose label for the entire video (e.g., 5s), we densely label each *short segment* (e.g., 1s). This fine-grained annotation prevents optimizing video frames with artifacts and greatly improves data efficiency.

Ziyi Wu's profile picture
Ziyi Wu1 year ago

*VLM dense labels* Prior works show that VLMs such as GPT struggle with long videos. We show that GPT is actually good at assessing short clips, achieving preference accuracy similar to task-specifically fine-tuned video reward models and DPO results close to using human labels.

Ziyi Wu's profile picture
Ziyi Wu1 year ago

Takeaways: - Videos are different from images due to the temporal axis; - We really need fine-grained supervision to improve video generation, e.g., the spatio-temporal motion repr in VideoJAM, the dense temporal captions in my previous work MinT, and the dense preferences here.

Related Videos

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:
1:31

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

DeepLearning.AI

16,779 views • 1 year ago

New podcast/lecture combo -- a case study in the messy details of Olmo 3 post training & DPO with Scott Geng. It's rare to make time for these discussions, but we cover: What it takes for a research idea to make it into a (near) frontier model. The messy side of DPO (usually data). Organizational challenges in multi-stage post-training recipes. Reflections on where research is heading today, and how people should think of DPO. DPO fundamentals (once more) Other topics. Scott is one of the great student's I've had the pleasure of working with for a few years! Chapters: 00:00 Introduction 04:32 Preference Tuning Basics 08:21 The DPO-Algorithm Zoo 17:06 Scaling Preference Data 21:55 Delta Learning Hypothesis 32:30: Applying it to Olmo 3 36:08 Challenge 1: Data Details 47:17 Challenge 2: Organization Management 51:45 Challenge 3: Scaling Issues 59:15 The Future of Academic Research Excited to keep making this content under the motivation of my book. Really plugging away now :)
1:06:30

New podcast/lecture combo -- a case study in the messy details of Olmo 3 post training & DPO with Scott Geng. It's rare to make time for these discussions, but we cover: What it takes for a research idea to make it into a (near) frontier model. The messy side of DPO (usually data). Organizational challenges in multi-stage post-training recipes. Reflections on where research is heading today, and how people should think of DPO. DPO fundamentals (once more) Other topics. Scott is one of the great student's I've had the pleasure of working with for a few years! Chapters: 00:00 Introduction 04:32 Preference Tuning Basics 08:21 The DPO-Algorithm Zoo 17:06 Scaling Preference Data 21:55 Delta Learning Hypothesis 32:30: Applying it to Olmo 3 36:08 Challenge 1: Data Details 47:17 Challenge 2: Organization Management 51:45 Challenge 3: Scaling Issues 59:15 The Future of Academic Research Excited to keep making this content under the motivation of my book. Really plugging away now :)

Nathan Lambert

25,036 views • 2 months ago

Loopy Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency paper page: With the introduction of diffusion-based video generation techniques, audio-conditioned human video generation has recently achieved significant breakthroughs in both the naturalness of motion and the synthesis of portrait details. Due to the limited control of audio signals in driving human motion, existing methods often add auxiliary spatial signals to stabilize movements, which may compromise the naturalness and freedom of motion. In this paper, we propose an end-to-end audio-only conditioned video diffusion model named Loopy. Specifically, we designed an inter- and intra-clip temporal module and an audio-to-latents module, enabling the model to leverage long-term motion information from the data to learn natural motion patterns and improving audio-portrait movement correlation. This method removes the need for manually specified spatial motion templates used in existing methods to constrain motion during inference. Extensive experiments show that Loopy outperforms recent audio-driven portrait diffusion models, delivering more lifelike and high-quality results across various scenarios.
0:45

Loopy Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency paper page: With the introduction of diffusion-based video generation techniques, audio-conditioned human video generation has recently achieved significant breakthroughs in both the naturalness of motion and the synthesis of portrait details. Due to the limited control of audio signals in driving human motion, existing methods often add auxiliary spatial signals to stabilize movements, which may compromise the naturalness and freedom of motion. In this paper, we propose an end-to-end audio-only conditioned video diffusion model named Loopy. Specifically, we designed an inter- and intra-clip temporal module and an audio-to-latents module, enabling the model to leverage long-term motion information from the data to learn natural motion patterns and improving audio-portrait movement correlation. This method removes the need for manually specified spatial motion templates used in existing methods to constrain motion during inference. Extensive experiments show that Loopy outperforms recent audio-driven portrait diffusion models, delivering more lifelike and high-quality results across various scenarios.

AK

128,912 views • 2 years ago

AI only exists today because it is trained on real data. The better the data input, The better the AI output. We capture incredibly high-quality raw data of real humans, in 3D, from all angles. You do the Math ✍🏻 Our data can vastly improve and expand the capabilities of AI
1:04

AI only exists today because it is trained on real data. The better the data input, The better the AI output. We capture incredibly high-quality raw data of real humans, in 3D, from all angles. You do the Math ✍🏻 Our data can vastly improve and expand the capabilities of AI

Metahero.io

24,545 views • 2 years ago

We (Pantheon) have been working on a robotics data quality pipeline that uncovered a series of major problems in public robotics datasets, especially for world modeling. To improve the quality of data available to open-source robotics, we're publishing annotations for four of the most popular datasets. Some examples of issues, and our report 🧵
0:30

We (Pantheon) have been working on a robotics data quality pipeline that uncovered a series of major problems in public robotics datasets, especially for world modeling. To improve the quality of data available to open-source robotics, we're publishing annotations for four of the most popular datasets. Some examples of issues, and our report 🧵

calix

182,282 views • 1 month ago

The First 100 begins now. Verify Quality is live on PrismaX. For the first time, anyone can score the robot training data that models learn from, earn points, and compete to become one of The First 100. Better data. Better models. The standard starts with you.
0:34

The First 100 begins now. Verify Quality is live on PrismaX. For the first time, anyone can score the robot training data that models learn from, earn points, and compete to become one of The First 100. Better data. Better models. The standard starts with you.

PrismaX

234,490 views • 4 months ago

Language following is a tough problem for VLAs: while these models can follow complex language, in practice getting datasets that enable language following is hard. We developed a method to counterfactually and automatically label data to improve language following! 🧵👇
1:12

Language following is a tough problem for VLAs: while these models can follow complex language, in practice getting datasets that enable language following is hard. We developed a method to counterfactually and automatically label data to improve language following! 🧵👇

Sergey Levine

44,287 views • 1 year ago

Introducing the #WELL3Ring: Elevating Health Data to On-Chain Assets🔘 At #WELL3, we are leading a transformative shift in how health data is valued and monetized. We cordially invite our #WELLID holders to be the first to experience the benefits of this DePIN wearable. By donning the WELL3 Ring and sharing your health data, you will gain access to unlimited and exclusive rewards🧵
0:40

Introducing the #WELL3Ring: Elevating Health Data to On-Chain Assets🔘 At #WELL3, we are leading a transformative shift in how health data is valued and monetized. We cordially invite our #WELLID holders to be the first to experience the benefits of this DePIN wearable. By donning the WELL3 Ring and sharing your health data, you will gain access to unlimited and exclusive rewards🧵

WELL3

223,060 views • 2 years ago

It is really hard to get robot policies that are both precise (small margins for error) and general (robust to env variations). We just released ViSk, where skin sensing is used to train fine-grained policies with ~1 hour of data. Below is a single-take video.
0:37

It is really hard to get robot policies that are both precise (small margins for error) and general (robust to env variations). We just released ViSk, where skin sensing is used to train fine-grained policies with ~1 hour of data. Below is a single-take video.

Lerrel Pinto

17,169 views • 2 years ago

Can LLMs invent better ways to train LLMs? At Sakana AI, we’re pioneering AI-driven methods to automate AI research and discovery. We’re excited to release DiscoPOP: a new SOTA preference optimization algorithm that was discovered and written by an LLM! Our method leverages LLMs to propose and implement new preference optimization algorithms. We then train models with those algorithms and evaluate their performance, providing feedback to the LLM. By repeating this process for multiple generations in an evolutionary loop, the LLM discovers many highly-performant and novel preference optimization objectives! Paper: GitHub: Model: We proudly collaborated with the University of Oxford (Foerster Lab for AI Research (now part of BOLD)) and Cambridge University (Mihaela van der Schaar) on this groundbreaking project. Looking ahead, we envision a future where AI-driven research reduces the need for extensive human intervention and computational resources. This will accelerate scientific discoveries and innovation, pushing the boundaries of what AI can achieve.
0:22

Can LLMs invent better ways to train LLMs? At Sakana AI, we’re pioneering AI-driven methods to automate AI research and discovery. We’re excited to release DiscoPOP: a new SOTA preference optimization algorithm that was discovered and written by an LLM! Our method leverages LLMs to propose and implement new preference optimization algorithms. We then train models with those algorithms and evaluate their performance, providing feedback to the LLM. By repeating this process for multiple generations in an evolutionary loop, the LLM discovers many highly-performant and novel preference optimization objectives! Paper: GitHub: Model: We proudly collaborated with the University of Oxford (Foerster Lab for AI Research (now part of BOLD)) and Cambridge University (Mihaela van der Schaar) on this groundbreaking project. Looking ahead, we envision a future where AI-driven research reduces the need for extensive human intervention and computational resources. This will accelerate scientific discoveries and innovation, pushing the boundaries of what AI can achieve.

Sakana AI

556,037 views • 2 years ago

Argus: an open-source robotics data annotation and quality pipeline. With new frontier VLMs like GPT-6 Astra, we can generate rich, high-quality annotations for robotics data for both training and dataset analysis. Argus delivers detailed, timestamp-level annotations while catching issues like mislabeled instructions, sped-up recordings, swapped camera streams, and unflagged operator mistakes. To support better data quality for the robotics community, we’re open-sourcing Argus for anyone to use 🧵
1:37

Argus: an open-source robotics data annotation and quality pipeline. With new frontier VLMs like GPT-6 Astra, we can generate rich, high-quality annotations for robotics data for both training and dataset analysis. Argus delivers detailed, timestamp-level annotations while catching issues like mislabeled instructions, sped-up recordings, swapped camera streams, and unflagged operator mistakes. To support better data quality for the robotics community, we’re open-sourcing Argus for anyone to use 🧵

calix

87,360 views • 25 days ago

🎆Chainbase is building the largest omnichain data network to provide high-quality data for the AGI era. Just like Oracle to Defi, Data to AI We aim to create a decentralized data economy and enhance the crypto world with advanced AI capabilities, where anyone can participate, contribute, and be incentivized. Explore the unique network features with us.🧵👇
0:52

🎆Chainbase is building the largest omnichain data network to provide high-quality data for the AGI era. Just like Oracle to Defi, Data to AI We aim to create a decentralized data economy and enhance the crypto world with advanced AI capabilities, where anyone can participate, contribute, and be incentivized. Explore the unique network features with us.🧵👇

Chainbase (💜,💛)

340,765 views • 2 years ago

OpenAI engineer explained every form of fine-tuning in 1 hour 46 minutes - better than $3000 fine-tuning bootcamps. pick your task -> try prompting first -> drop to SFT for style -> run DPO for preference -> run RFT for verifiable rewards -> ship a model 10x cheaper than GPT-5. That loop is why teams that fine-tune beat teams that only prompt on cost and speed. Prompting + SFT + DPO + RFT + OpenAI fine-tuning API - that's the stack. Watch and save it, then pick the right fine-tune for your task this week.
1:45:58

OpenAI engineer explained every form of fine-tuning in 1 hour 46 minutes - better than $3000 fine-tuning bootcamps. pick your task -> try prompting first -> drop to SFT for style -> run DPO for preference -> run RFT for verifiable rewards -> ship a model 10x cheaper than GPT-5. That loop is why teams that fine-tune beat teams that only prompt on cost and speed. Prompting + SFT + DPO + RFT + OpenAI fine-tuning API - that's the stack. Watch and save it, then pick the right fine-tune for your task this week.

h100envy

41,245 views • 3 months ago

Introducing Exa Connect: connecting agents to data beyond the public web. Available today with ZoomInfo, Crunchbase, Similarweb, and many other leading data providers.
1:00

Introducing Exa Connect: connecting agents to data beyond the public web. Available today with ZoomInfo, Crunchbase, Similarweb, and many other leading data providers.

Exa

1,361,870 views • 4 months ago

Introducing: Advanced Data Assets™ By LayerAI 🧬 Data is used to power Big Data and AI. The increasing demand for high quality data to improve AI models will lead to new companies creating value in the global data market. The problem: current data collection methods make data fall short of its true potential. Data is limited. Building a decentralized solution where users choose to productize and earn from their data opens untapped sources of income. 🌐 Creating Advanced Data Assets™ There exists a market opportunity to create a digital blockchain-based global data marketplace. Data collected using our nodes, our native network apps, settled on LayerAI. The Advanced Data. Meet the Ecosystem that empowers users to participate in the Data Economy.
0:32

Introducing: Advanced Data Assets™ By LayerAI 🧬 Data is used to power Big Data and AI. The increasing demand for high quality data to improve AI models will lead to new companies creating value in the global data market. The problem: current data collection methods make data fall short of its true potential. Data is limited. Building a decentralized solution where users choose to productize and earn from their data opens untapped sources of income. 🌐 Creating Advanced Data Assets™ There exists a market opportunity to create a digital blockchain-based global data marketplace. Data collected using our nodes, our native network apps, settled on LayerAI. The Advanced Data. Meet the Ecosystem that empowers users to participate in the Data Economy.

LayerAI | AI2Earn

27,140 views • 2 years ago

Introducing 𝐃𝐫𝐞𝐚𝐦𝐆𝐞𝐧! We got humanoid robots to perform totally new 𝑣𝑒𝑟𝑏𝑠 in new environments through video world models. We believe video world models will solve the data problem in robotics. Bringing the paradigm of scaling human hours to GPU hours. Quick 🧵
0:58

Introducing 𝐃𝐫𝐞𝐚𝐦𝐆𝐞𝐧! We got humanoid robots to perform totally new 𝑣𝑒𝑟𝑏𝑠 in new environments through video world models. We believe video world models will solve the data problem in robotics. Bringing the paradigm of scaling human hours to GPU hours. Quick 🧵

Joel Jang

117,849 views • 1 year ago

🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation. Compared to strong diffusion language models (DLMs) at similar parameter size, NCM achieves: ⚡ Up to 31× faster generation on downstream tasks like summarization and machine translation with comparable or better quality. 🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity and using 3–5× fewer training tokens. 🧩 Better performance on reasoning tasks like Sudoku, outperforming both AR and diffusion baselines with fewer sampling steps. With Ishan Khare, Iddah Mlauzi, Kelly Buchanan, Simran Arora, Azalia Mirhoseini, and Hazy Research: Strip Mall AI Research Club. More details in the thread 🧵👇
0:21

🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation. Compared to strong diffusion language models (DLMs) at similar parameter size, NCM achieves: ⚡ Up to 31× faster generation on downstream tasks like summarization and machine translation with comparable or better quality. 🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity and using 3–5× fewer training tokens. 🧩 Better performance on reasoning tasks like Sudoku, outperforming both AR and diffusion baselines with fewer sampling steps. With Ishan Khare, Iddah Mlauzi, Kelly Buchanan, Simran Arora, Azalia Mirhoseini, and Hazy Research: Strip Mall AI Research Club. More details in the thread 🧵👇

Hermann Kumbong

493,602 views • 12 days ago

Today we are introducing Dyna-2, a world-action model pre-trained on one million hours of human video. At this scale, for the first time, we discovered several new scaling laws: • world-action models exhibit scaling law on human data across four orders of magnitude, from 1000 to 1,000,000 hours, • this human data scaling law implied a scaling law on never seen robot data, • both data and objective matter; world modeling and scaling on video data are essential for cross-embodiment scaling transfer to emerge 🧵
1:42

Today we are introducing Dyna-2, a world-action model pre-trained on one million hours of human video. At this scale, for the first time, we discovered several new scaling laws: • world-action models exhibit scaling law on human data across four orders of magnitude, from 1000 to 1,000,000 hours, • this human data scaling law implied a scaling law on never seen robot data, • both data and objective matter; world modeling and scaling on video data are essential for cross-embodiment scaling transfer to emerge 🧵

Dyna Robotics

1,820,849 views • 2 months ago