Flow matching produces smooth, deterministic trajectories. In contrast, the... sampling process of a diffusion model is chaotic, resembling the random motion of gas particles.show more

Diffusion models leverage a variety of samplers. Deterministic methods... like DDIM produce orderly paths. In contrast, stochastic samplers like DDPM produce chaotic trajectories. Despite their differences, both methods draw valid samples from the underlying distribution.show more

Stochastic and deterministic sampling strategies for diffusion models produce... strikingly different trajectories, but both ultimately achieve the same aim. I had a great time presenting our work, Diffusion Explorer, this week at IEEE VIS in Vienna.show more

Laminar flow is a type of fluid (liquid or... gas) movement where the flow is smooth, consistent, and occurs in parallel layers with no disruption between them. It contrasts with turbulent flow, which is chaotic and irregular.show more

The Cucker-Smale system of ODEs is the simplest model... of particles exhibiting flocking behaviors. Diffusion of speeds is governed by a Laplacian on the fully connected graph of points.show more

In flow matching, a coupling determines how noise and... data samples are paired during training. The choice of coupling is important because it influences the geometry of trajectories at inference time. The simplest choice is the independent coupling, where noise and data points are paired arbitrarily. This can lead to curved trajectories as the model averages over many conflicting pairings. However, if we use optimal transport on batches of pairs, this leads to fewer ambiguous intersections that the model must resolve, leading to straighter trajectories at inference time.show more

Oldies but goldies: Felipe Cucker, Steve Smale, The Mathematics... of Emergence, 2007. Defines a simple model of particles exhibiting flocking behaviors. Diffusion of speeds is governed by a Laplacian on the fully connected graph of points.show more

Do you enjoy the process of carving wood shapes... that produces very smooth and precise shapes?show more

We have released the code and weights for our... #CVPR2023 paper "Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model"! code: abs: project: The demo is below:show more

The Dream 7B (diffusion reasoning language model) is OUT!... 🚨 I built a demo so you can test it out (and check the diffusion process live) 𖣯🔍show more

I've been putting together an interactive tool called DiffusionLab... for explaining the geometric intuition behind diffusion and flow based generative models. Sampling is actually being done in the browser using Tensorflow.js! It is still in the very early stages.show more

LLaDA (the first Large Language Diffusion Model) is *just*... out 💥 and I've built a demo, try out now 👨‍💻 It's mesmerizing to watch the diffusion process 🌀, and it being a diffusion model gives you superpowers like "the 4th word has to be pineapple" 🦸 Demo and weights 👇show more

Lorenz attractor is the set of limit trajectories for... a simplified mathematical model for atmospheric convection. It has a fractal geometry.show more

Random Circuit Sampling (RCS) is a necessary benchmark to... measure quantum computer performance in the presence of noise. Why is RCS considered a cornerstone of quantum computing? Find out in our latest breakdown. #QuantumAIshow more

I just spent weeks handwriting a 200+ page guide... on the entirety of Robotics foundations from scratch. Featuring: - Kinematics (Forward, Inverse Kin, Jacobians, DoF) - Controls (PID, Force/Impedance Control, Feedforward) - SLAM (Bayes Filters, Kalman, Particle Filters, Mapping) - Motion Planning (Dijkstra, A*, RRT, RRT*) - RL (MDP, Policy Gradients, PPO, Sim2Real, Value Func) - VLA (Q/K/V, ViT, SigLIP, Diffusion/Flow-Matching) - ACT, Diffusion Policy, UMI, LeRobot tooling - World Model (Dreamer, VideoGen, Gaussian Splatting) Everything to get started, for free.show more

We are also releasing self-contained lecture notes that explain... flow matching and diffusion models from scratch. This goes from "zero" to the state-of-the-art in modern Generative AI. 📖 Read the notes here: Joint work with Ezra Erives.show more

(1/2) Check out "𝐏𝐨𝐥𝐲𝐃𝐢𝐟𝐟: Generating 3D Polygonal Meshes with... Diffusion Models"! Our model operates directly on the polygons of 3D meshes and generates novel shapes as output through an iterative diffusion process.show more

The fine line between smooth motion and engine damage... is just a single strand of human hair.show more

The fine line between smooth motion and engine damage... is just a single strand of human hairshow more

🚀New paper out - We present Video-MSG (Multimodal Sketch... Guidance), a novel planning-based training-free guidance method for T2V models, improving control of spatial layout and object trajectories. 🔧 Key idea: • Generate a Video Sketch — a spatio-temporal plan with background, foreground, and motion in the pixel space. • Encode this structure directly into the latent space of the diffusion model during generation, which does not require fine-tuning or additional memory during inference. 🧵show more

An orrery is a mechanical model of the Solar... System, meticulously crafted to demonstrate the motion of planets around the Sun.show more