(1/n) Time to unify your favorite visual generative models, VLMs, and simulators for controllable visual generation—Introducing a Product of Experts (PoE) framework for inference-time knowledge composition from heterogeneous models.

(2/n) The composition yields better controllability and provides flexible user interfaces for specifying visual synthesis goals, enabling applications such as composing physics simulation into generated videos…

(3/n) …inserting graphics engine rendering into images, and more.

(4/n) PoE sampling is non-trivial in high dimensions. We adopt Annealed Importance Sampling, where particles are initially drawn from a simple base distribution and steered towards the target, with transition kernels computed from expert models.. Two possible annealing paths:

(5/5) Page: More details in paper: Team work with the incredible Carson Murtuza-Lanier, @zizhang_li, @du_yilun, and @jiajunwu_cs!

Join me as I put several Machine Learning models head-to-head to see which one can beat the market and deliver strong returns. In this free Substack post I share several models that deliver better returns with much lower drawdown compared to Buy-and-Hold approach.

Whats the best visual video generative models in your experience ?