Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

FramePack is out Packing Input Frame Context in Next-Frame Prediction Models for Video Generation

AK

428,014 subscribers

31,486 views • 1 year ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

2 Comments

AK1 year ago

discuss:

AK1 year ago

local app:

Related Videos

Can we simplify video generation by decomposing it into interleaved text-video co-generation? Would explicit, repeated thinking in language improve generation in pixels? We introduce TV2TV: a unified model that jointly learns - language modeling (next-token prediction) - video flow matching (next-frame prediction) At inference, TV2TV dynamically alternates between textual thinking and video generation. Model generations below: interleaved text plans and video slices (~1–2s) are co-generated over time, conditioned on a single frame per sport. 📖

Can we simplify video generation by decomposing it into interleaved text-video co-generation? Would explicit, repeated thinking in language improve generation in pixels? We introduce TV2TV: a unified model that jointly learns - language modeling (next-token prediction) - video flow matching (next-frame prediction) At inference, TV2TV dynamically alternates between textual thinking and video generation. Model generations below: interleaved text plans and video slices (~1–2s) are co-generated over time, conditioned on a single frame per sport. 📖

Xiaochuang Han

31,749 views • 7 months ago

InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbing

InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbing

Purz.ai

16,711 views • 11 months ago

Half Sword is out now with #RTXOn. Enable DLSS 4 with Multi Frame Generation for maximum performance.

Half Sword is out now with #RTXOn. Enable DLSS 4 with Multi Frame Generation for maximum performance.

NVIDIA GeForce

54,345 views • 5 months ago

Seedance 2.0 Omni-ref → a cheat code for consistency 🔓 Choose any input - image or video. Build your story - frame to frame aligned. Short tutorial below👇

Seedance 2.0 Omni-ref → a cheat code for consistency 🔓 Choose any input - image or video. Build your story - frame to frame aligned. Short tutorial below👇

Framer 🇱🇹

19,432 views • 3 months ago

10/118 wins. Frame 1. Frame 2. Frame 3. Frame 4!!! Guess who the person in the video is. 100k for you if she replies your comment. Wrong answers only.

10/118 wins. Frame 1. Frame 2. Frame 3. Frame 4!!! Guess who the person in the video is. 100k for you if she replies your comment. Wrong answers only.

Sir Khaycee

14,580 views • 7 months ago

Jeff Dean says models hallucinate because their training data is “squishy.” But what’s in the context window is sharp, the exact text or video frame right in front of them. The goal is to bring more of the world into that context. Compression gives it intuition. Context gives it precision.

Jeff Dean says models hallucinate because their training data is “squishy.” But what’s in the context window is sharp, the exact text or video frame right in front of them. The goal is to bring more of the world into that context. Compression gives it intuition. Context gives it precision.

vitrupo

41,600 views • 3 months ago

🔥 Booom... shipped! 🚀 VIDEO GENERATION is now live on Scenario. 20 video models, plus world-class #VEO3. ▶️ Animate any image ✨ Auto-prompting, first & last frame control ⚖️ Compare models side by side The MOST PRODUCTIVE video gen UX out there. 👉

🔥 Booom... shipped! 🚀 VIDEO GENERATION is now live on Scenario. 20 video models, plus world-class #VEO3. ▶️ Animate any image ✨ Auto-prompting, first & last frame control ⚖️ Compare models side by side The MOST PRODUCTIVE video gen UX out there. 👉

Emm | scenario.com

379,933 views • 1 year ago

"A-frame" is out now :) official video on youtube

"A-frame" is out now :) official video on youtube

chelsea cutler

24,735 views • 1 year ago

“Davido’s frame is insignificant. The size of his frame does not depict how great he is. But Wizkid command respect, power and greatness which is why he represent a symbol of hope in this generation and the next to come.” -Tina Mba

“Davido’s frame is insignificant. The size of his frame does not depict how great he is. But Wizkid command respect, power and greatness which is why he represent a symbol of hope in this generation and the next to come.” -Tina Mba

Arthur🇳🇬🇬🇧🇸🇳

189,961 views • 1 year ago

Next-generation features in a timeless Ray-Ban frame. Get your #RayBanMeta now, pre-order at the link in bio.

Next-generation features in a timeless Ray-Ban frame. Get your #RayBanMeta now, pre-order at the link in bio.

Ray-Ban

21,038 views • 2 years ago

Announcing: Keyframe Conditioning. Drop in First Frame, Last Frame (any frame, really!) and create the exact video you were envisioning. Check out some of our favorite videos!

Announcing: Keyframe Conditioning. Drop in First Frame, Last Frame (any frame, really!) and create the exact video you were envisioning. Check out some of our favorite videos!

Haiper AI

18,405 views • 1 year ago

Video might be the next intelligence substrate. Strikingly, video models are beginning to exhibit the same emergent reasoning behaviors first observed in LLMs—multi-path search, self-correction, and layer specialization. We demystify video reasoning and show it doesn’t happen frame-by-frame, but along diffusion steps. 🔗 📄 So, what's next? ;)

Video might be the next intelligence substrate. Strikingly, video models are beginning to exhibit the same emergent reasoning behaviors first observed in LLMs—multi-path search, self-correction, and layer specialization. We demystify video reasoning and show it doesn’t happen frame-by-frame, but along diffusion steps. 🔗 📄 So, what's next? ;)

Zhongang Cai

70,759 views • 4 months ago

Ray2 Keyframes, Extend & Loop is now here. Create your visual story with precise frame-by-frame control, seamless transitions, spatial exploration and long-form video durations using #Ray2 Text-to-Video and Image-to-Video models in Dream Machine.

Ray2 Keyframes, Extend & Loop is now here. Create your visual story with precise frame-by-frame control, seamless transitions, spatial exploration and long-form video durations using #Ray2 Text-to-Video and Image-to-Video models in Dream Machine.

Luma

7,362,289 views • 1 year ago

Long video generation usually results in context increasing/scaling during chunk/frame-wise rollout. Considering context scaling may require context selection, we thus introduce the idea of MoE into long context modelling and propose Mixture of Contexts. All previous context/memory is considered while the chosen ones are computed in a data-driven manner. You can easily enjoy 7x compute savings.

Long video generation usually results in context increasing/scaling during chunk/frame-wise rollout. Considering context scaling may require context selection, we thus introduce the idea of MoE into long context modelling and propose Mixture of Contexts. All previous context/memory is considered while the chosen ones are computed in a data-driven manner. You can easily enjoy 7x compute savings.

Ceyuan Yang

22,141 views • 11 months ago

Experience Stellar Blade on PC with #RTXOn! DLSS 4 with Multi Frame Generation boosts frame rates for maximum performance. Learn more:

Experience Stellar Blade on PC with #RTXOn! DLSS 4 with Multi Frame Generation boosts frame rates for maximum performance. Learn more:

NVIDIA GeForce

32,238 views • 1 year ago

World of Tanks: HEAT is coming, featuring DLSS 4 with Multi Frame Generation to accelerate frame rates, and NVIDIA Reflex to reduce PC latency for more responsive gameplay.

World of Tanks: HEAT is coming, featuring DLSS 4 with Multi Frame Generation to accelerate frame rates, and NVIDIA Reflex to reduce PC latency for more responsive gameplay.

NVIDIA GeForce

35,833 views • 11 months ago

Goodnight, 𝕏..·˚ ༘ ☾ ⋆｡˚ ☄︎ Newscast about xAI, SpaceX, Starlink, Neuralink, 𝕏, and Grok. Created using the xAI Video API. I upscaled the final frame to use as the starting point for the next clip's generation, but it only helped marginally. However, I found a paper called "FreeInit: Bridging Initialization Gap in Video Diffusion Models" that I'm going to try tomorrow to fix the drifting problem. The approach: 1. FFT both images (anchor frame + last frame) 2. Low-pass filter on anchor (structure/color/layout) 3. High-pass filter on last frame (details/edges/motion) 4. Combine and IFFT back

Goodnight, 𝕏..·˚ ༘ ☾ ⋆｡˚ ☄︎ Newscast about xAI, SpaceX, Starlink, Neuralink, 𝕏, and Grok. Created using the xAI Video API. I upscaled the final frame to use as the starting point for the next clip's generation, but it only helped marginally. However, I found a paper called "FreeInit: Bridging Initialization Gap in Video Diffusion Models" that I'm going to try tomorrow to fix the drifting problem. The approach: 1. FFT both images (anchor frame + last frame) 2. Low-pass filter on anchor (structure/color/layout) 3. High-pass filter on last frame (details/edges/motion) 4. Combine and IFFT back

tetsuo

97,901 views • 5 months ago

FBC: Firebreak Major Update Breakpoint is out now on all platforms for free featuring path tracing and DLSS 4 with Multi Frame Generation. #RTXOn →

FBC: Firebreak Major Update Breakpoint is out now on all platforms for free featuring path tracing and DLSS 4 with Multi Frame Generation. #RTXOn →

NVIDIA GeForce

28,380 views • 9 months ago

Our research team just released Flex-Forcing: a video generation method that lets a single model switch between generation methods at inference time. Right now there are two main approaches to video generation. Bidirectional diffusion models attend to every frame at once, holding structure well at the cost of speed. Autoregressive models generate frame by frame, so they stream fast and scale to long clips, but accumulate error and drift over time. Flex-Forcing trains a single model to do both, letting you choose from the range at inference based on your compute budget.

Our research team just released Flex-Forcing: a video generation method that lets a single model switch between generation methods at inference time. Right now there are two main approaches to video generation. Bidirectional diffusion models attend to every frame at once, holding structure well at the cost of speed. Autoregressive models generate frame by frame, so they stream fast and scale to long clips, but accumulate error and drift over time. Flex-Forcing trains a single model to do both, letting you choose from the range at inference based on your compute budget.

NVIDIA AI

31,412 views • 16 days ago