Alec Helbling's banner

Alec Helbling

@alec_helbling • 11,991 subscribers

Interpretability, Multimodality, Diffusion. Research Scientist @MITIBMLab, PhDing @GeorgiaTech.

Shorts

Stokes' Theorem is a classic result from vector calculus. It tells us that the line integral of a vector field over a loop is equal to the surface integral of the curl of the vector field over some enclosed surface.

Stokes' Theorem is a classic result from vector calculus. It tells us that the line integral of a vector field over a loop is equal to the surface integral of the curl of the vector field over some enclosed surface.

439,543 次观看

The Helmholtz decomposition is one of the fundamental results of vector calculus. It says any well-behaved vector field can be split into two parts, one capturing sources and sinks through divergence, and one capturing rotation through curl.

The Helmholtz decomposition is one of the fundamental results of vector calculus. It says any well-behaved vector field can be split into two parts, one capturing sources and sinks through divergence, and one capturing rotation through curl.

229,242 次观看

A while back I made a blog about rectified flows. It is quite relevant to this post.

A while back I made a blog about rectified flows. It is quite relevant to this post.

58,505 次观看

Flow matching produces smooth, deterministic trajectories. In contrast, the sampling process of a diffusion model is chaotic, resembling the random motion of gas particles.

Flow matching produces smooth, deterministic trajectories. In contrast, the sampling process of a diffusion model is chaotic, resembling the random motion of gas particles.

437,319 次观看

Flow Reasoning Models are looped models. In our recent work, we turn a flow model into a recurrent model to solve reasoning problems (e.g. Sudoku). Predictions are fed back as inputs and iteratively refined. Each pass gets its own local loss. No backprop through time.

Flow Reasoning Models are looped models. In our recent work, we turn a flow model into a recurrent model to solve reasoning problems (e.g. Sudoku). Predictions are fed back as inputs and iteratively refined. Each pass gets its own local loss. No backprop through time.

33,914 次观看

Stochastic and deterministic sampling strategies for diffusion models produce strikingly different trajectories, but both ultimately achieve the same aim. I had a great time presenting our work, Diffusion Explorer, this week at IEEE VIS in Vienna.

Stochastic and deterministic sampling strategies for diffusion models produce strikingly different trajectories, but both ultimately achieve the same aim. I had a great time presenting our work, Diffusion Explorer, this week at IEEE VIS in Vienna.

279,431 次观看

A curious property of diffusion models: when a sample is mostly noise, the model’s best guess is close to the mean. So early on samples can be pulled toward the center of the dataset, even when no data exists there, before rebounding toward the data manifold.

A curious property of diffusion models: when a sample is mostly noise, the model’s best guess is close to the mean. So early on samples can be pulled toward the center of the dataset, even when no data exists there, before rebounding toward the data manifold.

31,986 次观看

Hamiltonian Monte Carlo frames sampling from a probability distribution as a physics problem. By endowing "particles" with momentum and simulating their energy and motion through Hamilton's equations you can efficiently explore a distribution.

Hamiltonian Monte Carlo frames sampling from a probability distribution as a physics problem. By endowing "particles" with momentum and simulating their energy and motion through Hamilton's equations you can efficiently explore a distribution.

169,063 次观看

In flow matching, a coupling determines how noise and data samples are paired during training. The choice of coupling is important because it influences the geometry of trajectories at inference time. The simplest choice is the independent coupling, where noise and data points are paired arbitrarily. This can lead to curved trajectories as the model averages over many conflicting pairings. However, if we use optimal transport on batches of pairs, this leads to fewer ambiguous intersections that the model must resolve, leading to straighter trajectories at inference time.

In flow matching, a coupling determines how noise and data samples are paired during training. The choice of coupling is important because it influences the geometry of trajectories at inference time. The simplest choice is the independent coupling, where noise and data points are paired arbitrarily. This can lead to curved trajectories as the model averages over many conflicting pairings. However, if we use optimal transport on batches of pairs, this leads to fewer ambiguous intersections that the model must resolve, leading to straighter trajectories at inference time.

65,798 次观看

Flow Matching aims to learn a "flow" that transforms a simple source distribution (e.g. Gaussian) to an arbitrarily complex target distribution. This video shows the evolution of the marginal probability path as a source distribution is transformed to a target distribution.

Flow Matching aims to learn a "flow" that transforms a simple source distribution (e.g. Gaussian) to an arbitrarily complex target distribution. This video shows the evolution of the marginal probability path as a source distribution is transformed to a target distribution.

150,891 次观看

Text-to-image diffusion transformer models learn to align text and image representations as a byproduct of their conditional denoising task. By taking the dot product between the text and image representations of a DiT model (like Flux 2), you can create rich saliency maps.

Text-to-image diffusion transformer models learn to align text and image representations as a byproduct of their conditional denoising task. By taking the dot product between the text and image representations of a DiT model (like Flux 2), you can create rich saliency maps.

94,195 次观看

Functions are Vectors This is a great blog post from Max Slater about how viewing functions as infinite dimensional vectors unlocks powerful tools from linear algebra. I think this is an under-emphasized perspective in intro linear algebra classes.

Functions are Vectors This is a great blog post from Max Slater about how viewing functions as infinite dimensional vectors unlocks powerful tools from linear algebra. I think this is an under-emphasized perspective in intro linear algebra classes.

93,596 次观看

Many dimensionality reduction algorithms share a few central principles. 1. Construct a graph that captures the data's local structure 2. Measure "geodesic" distances between points using the graph 3. Project the points to a lower dimension while preserving these distances

Many dimensionality reduction algorithms share a few central principles. 1. Construct a graph that captures the data's local structure 2. Measure "geodesic" distances between points using the graph 3. Project the points to a lower dimension while preserving these distances

88,874 次观看

I've been putting together an interactive tool called DiffusionLab for explaining the geometric intuition behind diffusion and flow based generative models. Sampling is actually being done in the browser using Tensorflow.js! It is still in the very early stages.

I've been putting together an interactive tool called DiffusionLab for explaining the geometric intuition behind diffusion and flow based generative models. Sampling is actually being done in the browser using Tensorflow.js! It is still in the very early stages.

104,731 次观看

Diffusion models leverage a variety of samplers. Deterministic methods like DDIM produce orderly paths. In contrast, stochastic samplers like DDPM produce chaotic trajectories. Despite their differences, both methods draw valid samples from the underlying distribution.

Diffusion models leverage a variety of samplers. Deterministic methods like DDIM produce orderly paths. In contrast, stochastic samplers like DDPM produce chaotic trajectories. Despite their differences, both methods draw valid samples from the underlying distribution.

103,077 次观看

Langevin Monte Carlo allows you to draw samples from a probability distribution using its log gradient ∇ log p(x). By performing a sort of gradient ascent with noise you can navigate around the distribution. Langevin MC is heavily related to modern diffusion models.

Langevin Monte Carlo allows you to draw samples from a probability distribution using its log gradient ∇ log p(x). By performing a sort of gradient ascent with noise you can navigate around the distribution. Langevin MC is heavily related to modern diffusion models.

97,001 次观看

I added a guide to the repository for my project ManimML, which is a tool for visualizing neural network architectures and algorithms using only python code. It is built on top of @manim_community which was started by Grant Sanderson. Link:

I added a guide to the repository for my project ManimML, which is a tool for visualizing neural network architectures and algorithms using only python code. It is built on top of @manim_community which was started by Grant Sanderson. Link:

172,253 次观看

Stein Variational Gradient Descent (SVGD) is a particle-based approach to variational inference. SVGD iteratively transports a set of particles to match a target distribution by applying a form of functional gradient descent that minimizes the KL divergence.

Stein Variational Gradient Descent (SVGD) is a particle-based approach to variational inference. SVGD iteratively transports a set of particles to match a target distribution by applying a form of functional gradient descent that minimizes the KL divergence.

28,368 次观看

One of the simplest algorithms for sampling from a probability distribution is Random Walk Metropolis-Hastings. It proposes new samples by taking Gaussian-distributed steps, accepting or rejecting them to maintain the target distribution. I call this pdf the "fidget spinner".

One of the simplest algorithms for sampling from a probability distribution is Random Walk Metropolis-Hastings. It proposes new samples by taking Gaussian-distributed steps, accepting or rejecting them to maintain the target distribution. I call this pdf the "fidget spinner".

79,948 次观看

I made a tool called Diffusion Explorer that lets you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇

I made a tool called Diffusion Explorer that lets you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇

73,188 次观看

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Introducing Flow Reasoning Models. We developed a recurrent flow-based architecture to efficiently solve structured reasoning problems (e.g., Sudoku). FRMs apply continuous flows to discrete data and recurrently refine their past mistakes through self-conditioning.

Introducing Flow Reasoning Models. We developed a recurrent flow-based architecture to efficiently solve structured reasoning problems (e.g., Sudoku). FRMs apply continuous flows to discrete data and recurrently refine their past mistakes through self-conditioning.

129,722 次观看 • 1 个月前

Random-walk MC is a simple yet powerful way to sample from a distribution. But it produces highly correlated samples resulting in slow exploration. Hamiltonian Monte Carlo uses the distributions structure to propose farther-away candidates and explore much more efficiently.

Random-walk MC is a simple yet powerful way to sample from a distribution. But it produces highly correlated samples resulting in slow exploration. Hamiltonian Monte Carlo uses the distributions structure to propose farther-away candidates and explore much more efficiently.

92,097 次观看 • 1 个月前

KV caching is the fundamental optimization underpinning autoregressive LLM inference. Transformer layers store keys and values from earlier tokens, then reuse them as new tokens are generated. This avoids recomputation, but at the cost of memory capacity and bandwidth.

KV caching is the fundamental optimization underpinning autoregressive LLM inference. Transformer layers store keys and values from earlier tokens, then reuse them as new tokens are generated. This avoids recomputation, but at the cost of memory capacity and bandwidth.

36,668 次观看 • 1 个月前

Data often lie on a low-dimensional manifold embedded in a high-dimensional space. But these manifolds are often highly non-linear, making linear dimensionality reduction methods like PCA insufficient. This has motivated the development of non-linear dimensionality reduction.

Data often lie on a low-dimensional manifold embedded in a high-dimensional space. But these manifolds are often highly non-linear, making linear dimensionality reduction methods like PCA insufficient. This has motivated the development of non-linear dimensionality reduction.

212,812 次观看 • 11 个月前

I wrote an interactive article explaining the geometric intuition behind Rectified Flows. I visually explain why flow-models tend to learn curved trajectories, why this is bad for sampling latency, and a relatively simple technique for mitigating it. Check it out! Link 👇

I wrote an interactive article explaining the geometric intuition behind Rectified Flows. I visually explain why flow-models tend to learn curved trajectories, why this is bad for sampling latency, and a relatively simple technique for mitigating it. Check it out! Link 👇

159,423 次观看 • 9 个月前

An interesting phenomenon in dynamical systems is the limit cycle. A limit cycle is an isolated periodic trajectory, often taking form as a self-sustaining oscillation, where a system’s state follows a closed path and repeatedly returns to the same values.

An interesting phenomenon in dynamical systems is the limit cycle. A limit cycle is an isolated periodic trajectory, often taking form as a self-sustaining oscillation, where a system’s state follows a closed path and repeatedly returns to the same values.

63,594 次观看 • 5 个月前

Marching squares is a classic algorithm for constructing contours of a 2D scalar field. It is used to create visualizations like topographic maps. Grid cells are matched to simple polygons based on which of their corners are above or below a threshold.

Marching squares is a classic algorithm for constructing contours of a 2D scalar field. It is used to create visualizations like topographic maps. Grid cells are matched to simple polygons based on which of their corners are above or below a threshold.

89,660 次观看 • 8 个月前

Masked diffusion is a discrete analog of continuous diffusion. Continuous diffusion gradually adds Gaussian noise, then learns to reverse that process to recover data. Masked diffusion instead progressively replaces tokens with a mask token, then learns to fill them back in.

Masked diffusion is a discrete analog of continuous diffusion. Continuous diffusion gradually adds Gaussian noise, then learns to reverse that process to recover data. Masked diffusion instead progressively replaces tokens with a mask token, then learns to fill them back in.

24,757 次观看 • 2 个月前

One of the strengths of flow matching is that it decouples generative modeling from a specific noise process and choice of source distribution. You can select from a variety of initial distributions like a Gaussian or a uniform distribution and still train a valid flow model.

One of the strengths of flow matching is that it decouples generative modeling from a specific noise process and choice of source distribution. You can select from a variety of initial distributions like a Gaussian or a uniform distribution and still train a valid flow model.

48,705 次观看 • 8 个月前

A cool concept in topological data analysis is the simplicial complex. It connects points with edges, triangles, and higher order analogs. By gradually increasing each point's neighborhood size it captures data's topological structure (e.g. voids, holes) at different scales.

A cool concept in topological data analysis is the simplicial complex. It connects points with edges, triangles, and higher order analogs. By gradually increasing each point's neighborhood size it captures data's topological structure (e.g. voids, holes) at different scales.

54,499 次观看 • 11 个月前

Our work ConceptAttention was accepted to ICML 2025 as a Spotlight Poster ("top" 2.6% of submissions)! ConceptAttention creates rich saliency maps of text concepts present in generated images and videos. It requires no additional training, only repurposing existing parameters.

Our work ConceptAttention was accepted to ICML 2025 as a Spotlight Poster ("top" 2.6% of submissions)! ConceptAttention creates rich saliency maps of text concepts present in generated images and videos. It requires no additional training, only repurposing existing parameters.

60,859 次观看 • 1 年前

How can you identify the topological structure of data from finite samples? Persistent homology creates graph-like structures to capture topological features of data at different scales. Features that "persist" over many scales represent data's true structure rather than noise.

How can you identify the topological structure of data from finite samples? Persistent homology creates graph-like structures to capture topological features of data at different scales. Features that "persist" over many scales represent data's true structure rather than noise.

38,090 次观看 • 11 个月前

Class conditioning doesn’t just add new capabilities to flow models. It can make the learning problem easier. When there are multiple plausible paths, the class label tells the model which one to take, reducing errors from averaging over conflicting trajectories.

Class conditioning doesn’t just add new capabilities to flow models. It can make the learning problem easier. When there are multiple plausible paths, the class label tells the model which one to take, reducing errors from averaging over conflicting trajectories.

21,016 次观看 • 5 个月前

A visualization of a flow model that transforms a 1D standard Gaussian into a Gaussian mixture. The heatmap shows the changing density and the white lines reflect the trajectories of individual samples over time.

A visualization of a flow model that transforms a 1D standard Gaussian into a Gaussian mixture. The heatmap shows the changing density and the white lines reflect the trajectories of individual samples over time.

29,111 次观看 • 9 个月前

Introducing ConceptAttention, an approach to interpreting diffusion transformer models! Write a prompt, choose some concepts, generate an image, and get high-quality heatmaps of text concepts. Our method outperforms existing methods like cross attention. Link to demo 👇

Introducing ConceptAttention, an approach to interpreting diffusion transformer models! Write a prompt, choose some concepts, generate an image, and get high-quality heatmaps of text concepts. Our method outperforms existing methods like cross attention. Link to demo 👇

36,696 次观看 • 1 年前

Introducing ClickDiffusion! We developed a system for precise image manipulation and generation that combines natural language instructions with visual feedback provided by the user through a direct manipulation interface.

Introducing ClickDiffusion! We developed a system for precise image manipulation and generation that combines natural language instructions with visual feedback provided by the user through a direct manipulation interface.

36,315 次观看 • 2 年前

没有更多内容可加载