Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcing

AK

488,183 subscribers

19,381 views • 1 year ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

2 Comments

AK1 year ago

discuss with author:

Rainmaker2 years ago

In this free Substack post I share code for several machine learning models and engage in hyperparameter tuning that yields a model that delivers superior returns in the Gold market.

Related Videos

Our research team just released Flex-Forcing: a video generation method that lets a single model switch between generation methods at inference time. Right now there are two main approaches to video generation. Bidirectional diffusion models attend to every frame at once, holding structure well at the cost of speed. Autoregressive models generate frame by frame, so they stream fast and scale to long clips, but accumulate error and drift over time. Flex-Forcing trains a single model to do both, letting you choose from the range at inference based on your compute budget.

Our research team just released Flex-Forcing: a video generation method that lets a single model switch between generation methods at inference time. Right now there are two main approaches to video generation. Bidirectional diffusion models attend to every frame at once, holding structure well at the cost of speed. Autoregressive models generate frame by frame, so they stream fast and scale to long clips, but accumulate error and drift over time. Flex-Forcing trains a single model to do both, letting you choose from the range at inference based on your compute budget.

NVIDIA AI

31,412 views • 2 months ago

(1/n) Time to unify your favorite visual generative models, VLMs, and simulators for controllable visual generation—Introducing a Product of Experts (PoE) framework for inference-time knowledge composition from heterogeneous models.

(1/n) Time to unify your favorite visual generative models, VLMs, and simulators for controllable visual generation—Introducing a Product of Experts (PoE) framework for inference-time knowledge composition from heterogeneous models.

Yunzhi Zhang

49,161 views • 1 year ago

📢📢 "Align Your Flow: Scaling Continuous-Time Flow Map Distillation" New flow map framework for state-of-the-art few-step generation, w/ the amazing Amirmojtaba Sabour and Sanja Fidler. 🔥 Project page: 📜 Paper: 🧵Thread below... (1/n)

📢📢 "Align Your Flow: Scaling Continuous-Time Flow Map Distillation" New flow map framework for state-of-the-art few-step generation, w/ the amazing Amirmojtaba Sabour and Sanja Fidler. 🔥 Project page: 📜 Paper: 🧵Thread below... (1/n)

Karsten Kreis

21,659 views • 1 year ago

Struggling with slow inference of diffusion and flow models? Check out the video below—I’ve been using our new FastGen library to achieve 7-28x acceleration for text-2-image and {text,image,video}-2-video generation without sacrificing visual fidelity!

Struggling with slow inference of diffusion and flow models? Check out the video below—I’ve been using our new FastGen library to achieve 7-28x acceleration for text-2-image and {text,image,video}-2-video generation without sacrificing visual fidelity!

Julius Berner

13,764 views • 7 months ago

World models are moving beyond offline generation towards interactive, real-time experiences. Introducing ⚡FlashDreams⚡: an open-source high-performance inference and serving library built for autoregressive world models: 🔥 Up to 3.10× faster LingBot-World inference 🔥 Up to 2.12× faster Self-Forcing inference 🔥 Up to 1.40× faster Wan2.1 inference 🔥 8 integrated models 🔥 Multi-GPU, streaming, low-latency serving 🔥 Agentic skills that teach you how to use it FlashDreams is designed for a new generation of AI systems that continuously evolve over time while responding to user interactions. It powers applications across robotics, autonomous vehicle simulation, gaming, and virtual worlds. Github: Docs: Research page: Join the #flashdreams Discord channel at FlashDreams is also the runtime backbone behind NVIDIA OmniDreams ( 1/n #AI #WorldModels #FastInference #PhysicalAI #OpenSource #NVIDIA

World models are moving beyond offline generation towards interactive, real-time experiences. Introducing ⚡FlashDreams⚡: an open-source high-performance inference and serving library built for autoregressive world models: 🔥 Up to 3.10× faster LingBot-World inference 🔥 Up to 2.12× faster Self-Forcing inference 🔥 Up to 1.40× faster Wan2.1 inference 🔥 8 integrated models 🔥 Multi-GPU, streaming, low-latency serving 🔥 Agentic skills that teach you how to use it FlashDreams is designed for a new generation of AI systems that continuously evolve over time while responding to user interactions. It powers applications across robotics, autonomous vehicle simulation, gaming, and virtual worlds. Github: Docs: Research page: Join the #flashdreams Discord channel at FlashDreams is also the runtime backbone behind NVIDIA OmniDreams ( 1/n #AI #WorldModels #FastInference #PhysicalAI #OpenSource #NVIDIA

Ruilong Li

99,750 views • 3 months ago

⭐️New Paper⭐️ just in time for NeurIPS. Presenting: Test-time scaling of diffusions with flow maps. project: arXiv: Want to properly adapt your diffusion at inference time with a reward? You should do everything at the level of the flow map. With the amazing Amirmojtaba Sabour, Carles Domingo-Enrich, Nicholas Boffi, Sanja Fidler , Karsten Kreis, and Eric Vanden-Eijnden.

⭐️New Paper⭐️ just in time for NeurIPS. Presenting: Test-time scaling of diffusions with flow maps. project: arXiv: Want to properly adapt your diffusion at inference time with a reward? You should do everything at the level of the flow map. With the amazing Amirmojtaba Sabour, Carles Domingo-Enrich, Nicholas Boffi, Sanja Fidler , Karsten Kreis, and Eric Vanden-Eijnden.

Michael Albergo

28,319 views • 10 months ago

MindJourney Test-Time Scaling with World Models for Spatial Reasoning

MindJourney Test-Time Scaling with World Models for Spatial Reasoning

AK

61,826 views • 1 year ago

DigitalOcean Inference Router is now cache-aware. 🔀 Before, switching models mid-conversation meant losing your cache, and the savings and consistency it bought you. Now, Inference Router weighs that cost before switching. Set a model affinity header or routing budget for full control.

DigitalOcean Inference Router is now cache-aware. 🔀 Before, switching models mid-conversation meant losing your cache, and the savings and consistency it bought you. Now, Inference Router weighs that cost before switching. Set a model affinity header or routing budget for full control.

DigitalOcean

318,497 views • 1 month ago

Frontier models that use reasoning to "think" during inference are generating 5X more AI tokens per year. ✨ "Inference is now a thinking process. And in order to teach AI how to think, reinforcement learning and very significant computation was introduced into post-training," said NVIDIA CEO Jensen Huang in a recent keynote at #CES2026. Reinforcement learning is increasing computation demands across all AI scaling laws: pre-training, post-training, and test-time scaling. Learn more about reinforcement learning and AI scaling ➡️

Frontier models that use reasoning to "think" during inference are generating 5X more AI tokens per year. ✨ "Inference is now a thinking process. And in order to teach AI how to think, reinforcement learning and very significant computation was introduced into post-training," said NVIDIA CEO Jensen Huang in a recent keynote at #CES2026. Reinforcement learning is increasing computation demands across all AI scaling laws: pre-training, post-training, and test-time scaling. Learn more about reinforcement learning and AI scaling ➡️

NVIDIA AI Infrastructure

13,329 views • 8 months ago

Inference scaling part 1. Starting with a modded text generation function (temperature scaling, top-p filtering, multinomial sampling) to generate diverse outputs for self-consistency and best-of-N (improving answer accuracy by>2x) 00:00 Introduction and recap 00:31 Training-time and inference-time scaling 07:52 What we'll implement 11:47 Notebook setup and model loading 17:43 Building a flexible text generation function 24:40 Chain-of-thought prompting 28:26 Sampling and output diversity 33:43 Next-token logits and greedy decoding 38:20 Temperature scaling step by step 42:46 Softmax and token probabilities 47:42 Multinomial sampling 54:51 Adding temperature sampling to text generation 59:31 Top-p filtering step by step 1:10:23 Adding top-p filtering to text generation 1:13:43 Sampling and LLM watermarking 1:16:01 Self-consistency and majority voting 1:20:36 Implementing self-consistency 1:29:02 MATH-500 results 1:35:01 Accuracy and compute tradeoffs 1:36:50 Next steps and self-refinement

Inference scaling part 1. Starting with a modded text generation function (temperature scaling, top-p filtering, multinomial sampling) to generate diverse outputs for self-consistency and best-of-N (improving answer accuracy by>2x) 00:00 Introduction and recap 00:31 Training-time and inference-time scaling 07:52 What we'll implement 11:47 Notebook setup and model loading 17:43 Building a flexible text generation function 24:40 Chain-of-thought prompting 28:26 Sampling and output diversity 33:43 Next-token logits and greedy decoding 38:20 Temperature scaling step by step 42:46 Softmax and token probabilities 47:42 Multinomial sampling 54:51 Adding temperature sampling to text generation 59:31 Top-p filtering step by step 1:10:23 Adding top-p filtering to text generation 1:13:43 Sampling and LLM watermarking 1:16:01 Self-consistency and majority voting 1:20:36 Implementing self-consistency 1:29:02 MATH-500 results 1:35:01 Accuracy and compute tradeoffs 1:36:50 Next steps and self-refinement

Sebastian Raschka

44,158 views • 12 days ago

Introducing Modality Forcing, a recipe for post-training T2I models for SOTA RGB-Depth generation! Text-to-image (T2I) models learn rich representations of the spatial world. How do we build on this prior for high-quality depth generation? 🧵 [1/6]

Introducing Modality Forcing, a recipe for post-training T2I models for SOTA RGB-Depth generation! Text-to-image (T2I) models learn rich representations of the spatial world. How do we build on this prior for high-quality depth generation? 🧵 [1/6]

Bardienus Duisterhof @ IROS 🌉

66,913 views • 3 months ago

After absorbing the storm from Lec 16, I’m back in my comfort zone: chalk-n-talk. 😄 This time, I discussed Flow Matching models, the current SoTA for image/video generation. As usual, I built everything from first principles, equation by equation. This is the first time I’m putting my Flow Models lecture online. Have a look.

After absorbing the storm from Lec 16, I’m back in my comfort zone: chalk-n-talk. 😄 This time, I discussed Flow Matching models, the current SoTA for image/video generation. As usual, I built everything from first principles, equation by equation. This is the first time I’m putting my Flow Models lecture online. Have a look.

prathosh A P

79,834 views • 12 days ago

$META is betting on $AMD for AI inference. As AI models become more commoditized, total cost of ownership for token generation matters more than ever. $AMD GPUs are far more cost-efficient than $NVDA for inference. With 3.5B daily users ready to use its AI models, $META is going all-in on $AMD

$META is betting on $AMD for AI inference. As AI models become more commoditized, total cost of ownership for token generation matters more than ever. $AMD GPUs are far more cost-efficient than $NVDA for inference. With 3.5B daily users ready to use its AI models, $META is going all-in on $AMD

Daniel

20,558 views • 8 months ago

HandX Scaling Bimanual Motion and Interaction Generation paper:

HandX Scaling Bimanual Motion and Interaction Generation paper:

AK

10,562 views • 6 months ago

FLUX.2 [klein] 4B + 9B are the newest BFL models in the Flux family—combining image generation + image editing in one compact architecture. Built for interactive workflows and quick iteration, with extremely fast inference on distilled variants. Two models. Two modes. ⚡️

FLUX.2 [klein] 4B + 9B are the newest BFL models in the Flux family—combining image generation + image editing in one compact architecture. Built for interactive workflows and quick iteration, with extremely fast inference on distilled variants. Two models. Two modes. ⚡️

ComfyUI

19,899 views • 8 months ago

📢📢 "Proteina: Scaling Flow-based Protein Structure Generative Models" #ICLR2025 (Oral Presentation) 🔥 Project page: 📜 Paper: 🛠️ Code and weights: 🧵Details in thread... (1/n)

📢📢 "Proteina: Scaling Flow-based Protein Structure Generative Models" #ICLR2025 (Oral Presentation) 🔥 Project page: 📜 Paper: 🛠️ Code and weights: 🧵Details in thread... (1/n)

Karsten Kreis

42,470 views • 1 year ago

Demis Hassabis says despite the advent of models like DeepSeek, AI needs "more compute than ever" for experimentation and innovation, serving models and inference because "the more computing time you spend, you get more powerful and better answers"

Demis Hassabis says despite the advent of models like DeepSeek, AI needs "more compute than ever" for experimentation and innovation, serving models and inference because "the more computing time you spend, you get more powerful and better answers"

Tsarathustra

53,451 views • 1 year ago

Long video generation is a systems problem. Introducing LongLive-2.0 from NVIDIA Research: an end-to-end NVFP4 training and inference system for long video generation. Low-precision deployment often relies on post-training quantization, creating a gap between how models are trained and how they run. LongLive-2.0 aligns NVFP4-aware training, distillation, and W4A4 inference, maintaining strong benchmark quality while improving speed and memory efficiency.

Long video generation is a systems problem. Introducing LongLive-2.0 from NVIDIA Research: an end-to-end NVFP4 training and inference system for long video generation. Low-precision deployment often relies on post-training quantization, creating a gap between how models are trained and how they run. LongLive-2.0 aligns NVFP4-aware training, distillation, and W4A4 inference, maintaining strong benchmark quality while improving speed and memory efficiency.

NVIDIA AI

60,715 views • 4 months ago

What could scaling unlock for biology? Introducing ProGen3- our next AI foundation models for protein generation. We develop compute-optimal scaling laws up to 46B parameters on 1.5T tokens with real evidence in the wet lab. +we solve a new set of challenges for drug discovery

What could scaling unlock for biology? Introducing ProGen3- our next AI foundation models for protein generation. We develop compute-optimal scaling laws up to 46B parameters on 1.5T tokens with real evidence in the wet lab. +we solve a new set of challenges for drug discovery

Ali Madani

80,665 views • 1 year ago

Crusoe Managed Inference is now available. Run model inference faster, easier, and more cost-effectively. Large-context AI solutions demand fast outputs, massive throughput, and resilient scaling. Crusoe Managed Inference delivers breakthrough speed & reduced input-token spend.

Crusoe Managed Inference is now available. Run model inference faster, easier, and more cost-effectively. Large-context AI solutions demand fast outputs, massive throughput, and resilient scaling. Crusoe Managed Inference delivers breakthrough speed & reduced input-token spend.

Crusoe

12,096,815 views • 10 months ago