Stein Variational Gradient Descent (SVGD) is a particle-based approach... to variational inference. SVGD iteratively transports a set of particles to match a target distribution by applying a form of functional gradient descent that minimizes the KL divergence.show more

Langevin Monte Carlo allows you to draw samples from... a probability distribution using its log gradient ∇ log p(x). By performing a sort of gradient ascent with noise you can navigate around the distribution. Langevin MC is heavily related to modern diffusion models.show more

Oldies but goldies: Y Nesterov, A method for solving... a convex programming problem with convergence rate O(1/k^2). Improves the 1/k rate of vanilla gradient descent to 1/k^2.show more

Flow Matching aims to learn a "flow" that transforms... a simple source distribution (e.g. Gaussian) to an arbitrarily complex target distribution. This video shows the evolution of the marginal probability path as a source distribution is transformed to a target distribution.show more

Drones followed by a wave of missiles. Tehran is... applying the drone approach to disperse air defenses and facilitate hypersonic missile wave missions.show more

[VAE] by Hand ✍️ A Variational Auto Encoder (VAE)... learns the structure (mean and variance) of hidden features and generates new data from the learned structure. In contrast, GANs only learn to generate new data to fool a discriminator; they may not necessarily know the underlying structure of the data. The International Conference on Learning Representations (ICLR) this year announced its first ever "Test of Time Award" to recognizes the VAE paper, published 10 years ago. This exercise demonstrates how to calculate a VAE by hand. [1] Given: ↳ Three training examples X1, X2, X3 ↳ Copy training examples to the bottom ↳ The purpose is to train the network to reconstruct the training examples. ↳ Since each target is a training example itself, we use the Greek word "auto" which means "self." This crucial step is what makes an autoencoder "auto." [2] Encoder: Layer 1 + ReLU ↳ Multiply inputs with weights and biases ↳ Apply ReLU, crossing out negative values (-1 -> 0) [3] Encoder: Mean and Variance ↳ Multiply features with two sets of weights and biases ↳ 🟩 The first set predicts the means (𝜇) of latent distributions ↳ 🟪 The second set predicts the standard deviation (𝜎) of latent distributions [4] Reparameterization Trick: Random Offset ↳ Sample epsilon ε from the normal distribution with mean = 0 and variance = 1. ↳ The purpose is to randomly pick a offset away from the mean. ↳ Multiply the standard deviation values with epsilon values. ↳ The purpose is to scale the offset by the standard deviation. [5] Reparameterization Trick: Mean + Offset ↳ Add the sampled offset to predicted mean ↳ The result are new parameters or features 🟨 as inputs to the Decoder. [6] Decoder: Layer 1 + ReLU ↳ Multiply input features with weights and biases ↳ Apply ReLU, crossing out negative values. Here, -4 is crossed out. [7] Decoder: Layer 2 ↳ Multiply features with weights and biases ↳ The output is Decoder's attempt to reconstruct the input data X from reparameterized distributions described by 𝜇 and 𝜎. [8]-[10] KL Divergence Loss [8] Loss Gradient: Mean 𝜇 ↳ We want 𝜇 to approach 0. ↳ A lot of math called SGVB simplifies the calculation of loss gradients to simply 𝜇 [9,10] Loss Gradient: Stdev 𝜎 ↳ We want 𝜎 to approach 1. ↳ A lot of math simplifies the calculation to 𝜎 - (1/ 𝜎) [11] Reconstruction Loss ↳ We want the reconstructed data Y (dark 🟧) to be the same as the input data X. ↳ Some math involving Mean Square Error simplifies the calculation to Y - X.show more

In 2023, neuroscientists uncovered that our brain distinguishes reality... from imagination on a gradient, with a threshold marking the jump from one state to another. Learn about the biological breakthroughs of 2023:show more

I took album cover of GNX by Kendrick Lamar... and turned it into a 3D particle model in Threejs with TSL Used Grok 4 by xAI to improve the noise that moves particles.show more

Have you ever experienced such a high rate of... descent? If something is meant to happen, it will happen—Murphy's Law.show more

Rayen Bahri built a Blender addon around gradient (palette)... texturing, making it easier to set up and use this workflow directly inside Blender. 🎨 See the details and background of the tool: #b3d #3dartshow more

An up-close view of TriangleSplatting+. Still hard to believe... that there's a gradient leading to this dataset and that ADAM on GPUs is able to find it. Unthinkable just 5 years ago. Thanks Andrea Tagliasacchi 🇨🇦 for putting the PLY on the project page.show more

Saw a couple of character trail effects recently, so... wanted to try something using #Niagara mesh particles. Particle uploads bone transforms used by material for skinning vertices. #UE5show more

The Gaussian Wave Packet ✍️ It is a way... to describe a quantum particle, like an electron, that is located in a certain area of space instead of being spread out everywhere. It forms a bell-shaped curve, where the peak of the curve indicates the most likely place to find the particle. This shape is special because it offers the best way to pack a particle; it meets the physical limit set by the Heisenberg Uncertainty Principle, striking the best balance between knowing a particle's position and its momentum. However, because this packet consists of many different overlapping waves, it is naturally unstable for a free particle. Over time, these internal waves travel at slightly different speeds, which causes the bell curve to gradually flatten and widen. This process is known as dispersion. In simpler terms, the longer a quantum particle travels freely, the more unclear its exact location becomes. Video : Jeedecodeshow more

Zephyren makes a thunderous descent to Sonaria as the... newest disaster event creature dropping this Friday! What do you think of Zephyren? 🌪️ #CreaturesofSonaria #RobloxDevshow more

I'm trying to figure out where to place a... joystick's digital tween when you hold a real one. The current approach is to recreate a plane based on the positions of your intermediate and proximal joints.show more

This is absolutely genius. Nigel McGuinness *technically* applying a... hold while Josh Woods is on the ropes, thus forcing Woods to use one of his rope breaks in a Pure Rules match. Exhibit Z of McGuinness being a wrestling savant.show more

【#Video】"Wimpy" bungee jumping is now a thing! A "mild"... version of bungee jumping has gone viral at a scenic spot in Zhejiang Province. Unlike traditional free-fall jumps, this one uses a deceleration device to slow the descent, making it accessible to those who want the experience without the terror.show more

One of the simplest algorithms for sampling from a... probability distribution is Random Walk Metropolis-Hastings. It proposes new samples by taking Gaussian-distributed steps, accepting or rejecting them to maintain the target distribution. I call this pdf the "fidget spinner".show more