Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Gradient, Divergence and Curl Three vector fields, moving at the same time. Gradient: phi = 1 - x^2 - 2y^2, so grad phi = (-2x, -4y). Every tracer flows uphill and crosses each contour at a right angle. The red probe climbs to the top, and its arrow shrinks... to zero there, because the slope at a peak is zero. Divergence: F = (x, y) pushes out of every point, div F = 1 + 1 = 2 (a source). Flip it to F = (-x, -y) and div F = -2 (a sink): the flux through the circle turns around. Curl: F = (-y, x) turns everything around the centre, curl F = 1 - (-1) = 2, pointing out of the screen. A paddle wheel carried by the flow spins once per orbit. Reverse the field and the curl points into the screen, -2.show more

Matheorems

11,999 subscribers

36,836 views • 1 day ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

9 Comments

Juan Jeorge🐦✨🌎1 day ago

this stuff is so cool this is what math class should be

BUDDHA OF ENTIRETY1 day ago

super cool

. Jones1 day ago

@grok why is this important

marte43211 day ago

Me encanta!!!!,

신재구1 day ago

인류는 특이점을 돌파하고 있다.

entropic astrophysical united architecture1 day ago

hey Matheorems! has anyone that you that you are an idiot yet today?

bios(is ubiquitous)1 day ago

円形導波管のモードの何かだ

Sowrabhossain1 day ago

Test

Lőrinc Bauer1 day ago

nice

Related Videos

A square wave is a sum of odd sines, f(x) = (4/π) ∑ sin((2n−1)x)/(2n−1), and each term is a circle of radius 4/(π(2n−1)) turning 2n−1 times per period. Chained end to end, the height of the n-th joint is the partial sum Sₙ(x). The overshoot at each jump tends to 0.179, about 9 % of the jump, however many terms are added. At x = π/2 every arm points straight up or down and f = 1, which is Leibniz's series: π/4 = 1 − 1/3 + 1/5 − 1/7 + ⋯.

A square wave is a sum of odd sines, f(x) = (4/π) ∑ sin((2n−1)x)/(2n−1), and each term is a circle of radius 4/(π(2n−1)) turning 2n−1 times per period. Chained end to end, the height of the n-th joint is the partial sum Sₙ(x). The overshoot at each jump tends to 0.179, about 9 % of the jump, however many terms are added. At x = π/2 every arm points straight up or down and f = 1, which is Leibniz's series: π/4 = 1 − 1/3 + 1/5 − 1/7 + ⋯.

matharium

47,537 views • 1 month ago

Do you actually know what convex optimization is in the geometric, guarantee-theoretic sense or have you only met it through solvers and loss curves? Convexity is rare comfort in optimization...there are no spurious local minima, no surprise traps, and inequalities you can use like tools instead of prayers. So, what is this convexity? Let x = (x₁, x₂) and let f(x) be convex. Plot the surface z = f(x). Pick a contact point x₀. The local slope is the gradient p = ∇f(x₀). That p is exactly the data that defines the supporting plane: z = f(x₀) + p · (x − x₀). Thus, f is said to be convex because for every x, f(x) ≥ f(x₀) + p · (x − x₀). So the plane at x₀ can slide under the surface, but it never slices through it. Not near the point...everywhere. Now for here is the interesting part: The slope becomes a coordinate system! Rewrite the same plane as z = p · x − b, where b is the offset. Because the plane passes through (x₀, f(x₀)), the offset is forced to be b = p · x₀ − f(x₀). And that number isn’t just geometry trivia. It’s the convex conjugate: f*(p) = sup over x ( p · x − f(x) ). At a differentiable contact point, the supporting plane touches f tightly enough that the supremum is achieved at x₀, giving the identity f*(p) = p · x₀ − f(x₀) when p = ∇f(x₀). So one moving contact point gives two linked readouts: primal position x₀ dual position (slope) p = ∇f(x₀) dual offset f*(p) One surface. Two worlds. #ConvexOptimization #Optimization #MachineLearning #SignalProcessing #AppliedMath #Engineering

Do you actually know what convex optimization is in the geometric, guarantee-theoretic sense or have you only met it through solvers and loss curves? Convexity is rare comfort in optimization...there are no spurious local minima, no surprise traps, and inequalities you can use like tools instead of prayers. So, what is this convexity? Let x = (x₁, x₂) and let f(x) be convex. Plot the surface z = f(x). Pick a contact point x₀. The local slope is the gradient p = ∇f(x₀). That p is exactly the data that defines the supporting plane: z = f(x₀) + p · (x − x₀). Thus, f is said to be convex because for every x, f(x) ≥ f(x₀) + p · (x − x₀). So the plane at x₀ can slide under the surface, but it never slices through it. Not near the point...everywhere. Now for here is the interesting part: The slope becomes a coordinate system! Rewrite the same plane as z = p · x − b, where b is the offset. Because the plane passes through (x₀, f(x₀)), the offset is forced to be b = p · x₀ − f(x₀). And that number isn’t just geometry trivia. It’s the convex conjugate: f(p) = sup over x ( p · x − f(x) ). At a differentiable contact point, the supporting plane touches f tightly enough that the supremum is achieved at x₀, giving the identity f(p) = p · x₀ − f(x₀) when p = ∇f(x₀). So one moving contact point gives two linked readouts: primal position x₀ dual position (slope) p = ∇f(x₀) dual offset f*(p) One surface. Two worlds. #ConvexOptimization #Optimization #MachineLearning #SignalProcessing #AppliedMath #Engineering

Mathelirium

38,506 views • 10 months ago

There's a profound elegance in watching gradient descent find the signal in the noise. This is a simple MLP with sigmoid activations slowly warping to approximate f(x) = 1/(1 + x²). Watch the loss curve right around epoch 400. That sudden phase transition where the network finally "gets it" and snaps into the Witch of Agnesi shape is incredibly satisfying to watch.

There's a profound elegance in watching gradient descent find the signal in the noise. This is a simple MLP with sigmoid activations slowly warping to approximate f(x) = 1/(1 + x²). Watch the loss curve right around epoch 400. That sudden phase transition where the network finally "gets it" and snaps into the Witch of Agnesi shape is incredibly satisfying to watch.

Mathematica

78,616 views • 7 months ago

Ronaldinho curl is so special. He has 2 curl animations which is very rare. 1. tip of the foot curl. 2. inside foot curl.

Ronaldinho curl is so special. He has 2 curl animations which is very rare. 1. tip of the foot curl. 2. inside foot curl.

Bebe

18,781 views • 4 months ago

Indiana cover 2 replacement pressure vs Ohio State. Bring the nickel star tot he field. Drop the DE to the boundary. strong side-mike curl SS-deep 1/2, cb flat 2-1 Weak safety plays the middle hole The weak cb plays deep 1/2, de to the boundary has the curl and the weak LB has the flat. Great scheme and execution.

Indiana cover 2 replacement pressure vs Ohio State. Bring the nickel star tot he field. Drop the DE to the boundary. strong side-mike curl SS-deep 1/2, cb flat 2-1 Weak safety plays the middle hole The weak cb plays deep 1/2, de to the boundary has the curl and the weak LB has the flat. Great scheme and execution.

coachkou

26,790 views • 11 months ago

🔥🔥 OAU won UNIVERSITY DUEL FINAL against FUTA. Quayum at his best as usual ❤️ The 2nd to the last question here: Curl of the Grad of any scalar field is the 0 vector (DO-ROGO), and Div. of the curl of any vector field is also 0” – that’s basic MTH 202 at OAU💯 #GreatIfe #OAUX

🔥🔥 OAU won UNIVERSITY DUEL FINAL against FUTA. Quayum at his best as usual ❤️ The 2nd to the last question here: Curl of the Grad of any scalar field is the 0 vector (DO-ROGO), and Div. of the curl of any vector field is also 0” – that’s basic MTH 202 at OAU💯 #GreatIfe #OAUX

marviee

22,077 views • 6 months ago

Every LLM you've used runs on physics governed by divergence and curl. Maxwell's equations — the complete theory of light, electricity, and magnetism — are just four statements written in this language. One of them says the divergence of the magnetic field is zero everywhere, which is the exact mathematical reason magnetic monopoles can't exist. Divergence measures how much a vector field acts as a source or sink at a point; curl measures how much it rotates. That's it. But those two tools describe predator-prey population dynamics, the phase spaces of any differential equation, and the loss landscapes your GPU navigates during backpropagation — any system where a vector tells you which direction things want to move. 13 minutes. Bookmark & watch today.

Every LLM you've used runs on physics governed by divergence and curl. Maxwell's equations — the complete theory of light, electricity, and magnetism — are just four statements written in this language. One of them says the divergence of the magnetic field is zero everywhere, which is the exact mathematical reason magnetic monopoles can't exist. Divergence measures how much a vector field acts as a source or sink at a point; curl measures how much it rotates. That's it. But those two tools describe predator-prey population dynamics, the phase spaces of any differential equation, and the loss landscapes your GPU navigates during backpropagation — any system where a vector tells you which direction things want to move. 13 minutes. Bookmark & watch today.

Eron

25,369 views • 1 month ago

Gradient, Divergence and Curl

Gradient, Divergence and Curl

Math Files

46,422 views • 15 days ago

Has anyone ever come across this way of visualizing the convolution of two functions f and g: Look at the graph of the two-variable function f(x)g(y), and consider diagonal slices over the line x + y = k. The area of those slices represents (f * g)(k).

Has anyone ever come across this way of visualizing the convolution of two functions f and g: Look at the graph of the two-variable function f(x)g(y), and consider diagonal slices over the line x + y = k. The area of those slices represents (f * g)(k).

Grant Sanderson

397,419 views • 3 years ago

Ask anyone who’s taken a course in Ordinary Differential Equations (ODEs) what a solution to an ODE represents geometrically, and most of them won’t have a clean answer. When I first took ordinary differential equations, the pattern was always the same. Early on it turns into a speedrun of methods: separation of variables, integrating factors, variation of parameters, Bernoulli, exact equations. Then pretty quickly the course slides into hammer-picking. Spot the form, apply the recipe, move on. Too mechanical! And the real problem is what you don’t walk away with. You leave with a toolkit, but without a feel for what a differential equation even is, especially geometrically. That matters because in real modeling the equations you meet are rarely nice enough to reward memorised recipes. So you get trained to solve toy forms, while the actual subject stays blurry. The behavior. The flow. The shape of solutions. It wasn't until I watched the first lecture of Professor Arthur Mattuck that I realized I didn’t actually know what a solution to a differential equation represents geometrically. His point is almost embarrassingly simple. A first-order ODE is a slope field, and a solution is a curve that stays tangent to that field everywhere. The math breakdown: Write the ODE as dy/dx = f(x,y). At each point (x,y), attach a tiny line segment with slope f(x,y). A function y = y₁(x) is a solution exactly when its graph follows those slopes. At every x, the slope of the curve equals the slope prescribed by the field at the point on the curve. That’s the one line that ties both viewpoints together: y₁′(x) = f(x, y₁(x)). So solving the ODE and drawing an integral curve are the same statement in two languages. Once you see that, you stop obsessing over whether you can write y(x) in closed form. You start asking the questions that actually matter. Where do solutions flow. Where do they get trapped. Where do they blow up. Where does existence or uniqueness fail because the field isn’t even defined? That’s the perspective shift I wish every ODE course forces early. It’s also why I keep pairing math with animation. #DifferentialEquations #ODEs #VectorFields #AppliedMathematics #Mathematics #

Ask anyone who’s taken a course in Ordinary Differential Equations (ODEs) what a solution to an ODE represents geometrically, and most of them won’t have a clean answer. When I first took ordinary differential equations, the pattern was always the same. Early on it turns into a speedrun of methods: separation of variables, integrating factors, variation of parameters, Bernoulli, exact equations. Then pretty quickly the course slides into hammer-picking. Spot the form, apply the recipe, move on. Too mechanical! And the real problem is what you don’t walk away with. You leave with a toolkit, but without a feel for what a differential equation even is, especially geometrically. That matters because in real modeling the equations you meet are rarely nice enough to reward memorised recipes. So you get trained to solve toy forms, while the actual subject stays blurry. The behavior. The flow. The shape of solutions. It wasn't until I watched the first lecture of Professor Arthur Mattuck that I realized I didn’t actually know what a solution to a differential equation represents geometrically. His point is almost embarrassingly simple. A first-order ODE is a slope field, and a solution is a curve that stays tangent to that field everywhere. The math breakdown: Write the ODE as dy/dx = f(x,y). At each point (x,y), attach a tiny line segment with slope f(x,y). A function y = y₁(x) is a solution exactly when its graph follows those slopes. At every x, the slope of the curve equals the slope prescribed by the field at the point on the curve. That’s the one line that ties both viewpoints together: y₁′(x) = f(x, y₁(x)). So solving the ODE and drawing an integral curve are the same statement in two languages. Once you see that, you stop obsessing over whether you can write y(x) in closed form. You start asking the questions that actually matter. Where do solutions flow. Where do they get trapped. Where do they blow up. Where does existence or uniqueness fail because the field isn’t even defined? That’s the perspective shift I wish every ODE course forces early. It’s also why I keep pairing math with animation. #DifferentialEquations #ODEs #VectorFields #AppliedMathematics #Mathematics #

Mathelirium

40,936 views • 9 months ago

In 1996, James Sethian showed a remarkably clean idea. You can get shortest routes through a messy world by letting a wave expand once. No trial paths. No scanning beams. Just one growing front. You compute an arrival time field T(x,y). At each point, T means how long the front needs to reach that point. The rule is ‖∇T‖ = 1/F. If the medium is fast, meaning F is large, the front moves quickly and T grows slowly. If the medium is slow, meaning F is small, the front moves slowly and T grows quickly. Obstacles act like F is near zero, so the front cannot pass through and instead wraps around. Then you get the path without searching. Once T exists, you pick any start point and follow xdot proportional to minus grad T. You slide downhill on the time landscape and trace a fastest route back to the source. This is the Hamilton Jacobi and optimal control view that Tsitsiklis (1995) made precise. Compute the value or arrival time function first, and the optimal trajectories follow from it. #FastMarching #EikonalEquation #HamiltonJacobi #OptimalControl #ShortestPath #ComputationalGeometry

In 1996, James Sethian showed a remarkably clean idea. You can get shortest routes through a messy world by letting a wave expand once. No trial paths. No scanning beams. Just one growing front. You compute an arrival time field T(x,y). At each point, T means how long the front needs to reach that point. The rule is ‖∇T‖ = 1/F. If the medium is fast, meaning F is large, the front moves quickly and T grows slowly. If the medium is slow, meaning F is small, the front moves slowly and T grows quickly. Obstacles act like F is near zero, so the front cannot pass through and instead wraps around. Then you get the path without searching. Once T exists, you pick any start point and follow xdot proportional to minus grad T. You slide downhill on the time landscape and trace a fastest route back to the source. This is the Hamilton Jacobi and optimal control view that Tsitsiklis (1995) made precise. Compute the value or arrival time function first, and the optimal trajectories follow from it. #FastMarching #EikonalEquation #HamiltonJacobi #OptimalControl #ShortestPath #ComputationalGeometry

Mathelirium

203,228 views • 9 months ago

We just watched Professor Arthur Mattuck kick off MIT’s ODE course with the one interpretation that most differential equations classes somehow postpone: An ordinary differential equation isn’t primarily a method hunt. It’s a geometric rule. You write dy/dx = f(x,y), and that right-hand side is literally telling you the slope your solution curve must have at each point (x,y). So I built this animation as a visual companion to that first lecture. It draws the direction field (little line elements whose slope is f(x,y)) and then shows integral curves sliding through it...curves that are tangent to the field everywhere they go. Two quick examples from the animation: For dy/dx = −x/y, the slope field steers you onto circles x² + y² = R². You also see a subtle point that gets missed when everything is taught as y(x)...even when the curve exists smoothly, the graph y(x) may only exist on a limited interval (|x|<R for the upper semicircle). For dy/dx = 1 + x − y, the isoclines (curves where the slope is constant) make the global behavior obvious...trajectories get funneled into a corridor and become asymptotic to the special solution y=x. You learn qualitative behavior without solving it the traditional way. #DifferentialEquations #ODEs #MITOCW #VectorFields #MathAnimation #Mathematics

We just watched Professor Arthur Mattuck kick off MIT’s ODE course with the one interpretation that most differential equations classes somehow postpone: An ordinary differential equation isn’t primarily a method hunt. It’s a geometric rule. You write dy/dx = f(x,y), and that right-hand side is literally telling you the slope your solution curve must have at each point (x,y). So I built this animation as a visual companion to that first lecture. It draws the direction field (little line elements whose slope is f(x,y)) and then shows integral curves sliding through it...curves that are tangent to the field everywhere they go. Two quick examples from the animation: For dy/dx = −x/y, the slope field steers you onto circles x² + y² = R². You also see a subtle point that gets missed when everything is taught as y(x)...even when the curve exists smoothly, the graph y(x) may only exist on a limited interval (|x|<R for the upper semicircle). For dy/dx = 1 + x − y, the isoclines (curves where the slope is constant) make the global behavior obvious...trajectories get funneled into a corridor and become asymptotic to the special solution y=x. You learn qualitative behavior without solving it the traditional way. #DifferentialEquations #ODEs #MITOCW #VectorFields #MathAnimation #Mathematics

Mathelirium

22,497 views • 10 months ago

When I first took ordinary differential equations, the pattern was always the same. Week 1 turns into a speedrun of methods: separation of variables, integrating factors, variation of parameters, Bernoulli, exact equations… and by Week 2 or 3 the course has quietly degenerated into hammer-picking. Spot the form, apply the recipe, move on. Mechanical! Fuuuuck!😫😫😫😫 The problem is what you don’t walk away with. You leave with a toolkit, but without a feel for what a differential equation even is, especially geometrically. And that’s a big deal, because in real modeling the equations you meet are rarely nice enough to reward memorized recipes. So you end up trained to solve toy forms, while the actual subject...the behavior, the flow, the shape of solutions stays blurry. This is why I’m biased toward the old-timers. Their old-school way of doing things always surprises me:...they’ll spend time on one idea until it sticks, instead of sprinting through a syllabus checklist. One lecture from them and you start noticing a contrast. A lot of modern teaching feels like "finish the content,". You get marched through techniques, but you’re not left with a single thought that keeps bothering you later...the kind of thought that actually pushes you toward research-level curiosity. MIT OpenCourseWare’s Professor Arthur Mattuck did that to me in his very first ODE lecture. One lecture, and your whole relationship with dy/dx = f(x,y) changes. In this segment, Prof. Mattuck is basically saying: A first-order ODE is a slope field, and a solution is a curve that moves everywhere tangent to that field. The math breakdown Write the ODE as dy/dx = f(x,y). At each point (x,y) you attach a tiny line segment with slope f(x,y). A function y = y₁(x) is a solution exactly when its graph follows those slopes:. At every x, the slope of the curve equals the slope prescribed by the field at the point on the curve. That’s the single line that unifies both viewpoints: y₁′(x) = f(x, y₁(x)). So solving the ODE and drawing an integral curve are the same statement in two languages!👌🏻 Once you see that, you can stop obsessing over whether you can write y(x) in closed form. You can start asking the questions that matter: where do solutions flow, where do they get trapped, where do they blow up, and where does existence/uniqueness fail just because the field isn’t even defined? That’s the perspective shift I wish every ODE course forces early and it’s exactly why I keep pairing math with animation. #DifferentialEquations #ODEs #VectorFields #MathAnimation #Mathematics

When I first took ordinary differential equations, the pattern was always the same. Week 1 turns into a speedrun of methods: separation of variables, integrating factors, variation of parameters, Bernoulli, exact equations… and by Week 2 or 3 the course has quietly degenerated into hammer-picking. Spot the form, apply the recipe, move on. Mechanical! Fuuuuck!😫😫😫😫 The problem is what you don’t walk away with. You leave with a toolkit, but without a feel for what a differential equation even is, especially geometrically. And that’s a big deal, because in real modeling the equations you meet are rarely nice enough to reward memorized recipes. So you end up trained to solve toy forms, while the actual subject...the behavior, the flow, the shape of solutions stays blurry. This is why I’m biased toward the old-timers. Their old-school way of doing things always surprises me:...they’ll spend time on one idea until it sticks, instead of sprinting through a syllabus checklist. One lecture from them and you start noticing a contrast. A lot of modern teaching feels like "finish the content,". You get marched through techniques, but you’re not left with a single thought that keeps bothering you later...the kind of thought that actually pushes you toward research-level curiosity. MIT OpenCourseWare’s Professor Arthur Mattuck did that to me in his very first ODE lecture. One lecture, and your whole relationship with dy/dx = f(x,y) changes. In this segment, Prof. Mattuck is basically saying: A first-order ODE is a slope field, and a solution is a curve that moves everywhere tangent to that field. The math breakdown Write the ODE as dy/dx = f(x,y). At each point (x,y) you attach a tiny line segment with slope f(x,y). A function y = y₁(x) is a solution exactly when its graph follows those slopes:. At every x, the slope of the curve equals the slope prescribed by the field at the point on the curve. That’s the single line that unifies both viewpoints: y₁′(x) = f(x, y₁(x)). So solving the ODE and drawing an integral curve are the same statement in two languages!👌🏻 Once you see that, you can stop obsessing over whether you can write y(x) in closed form. You can start asking the questions that matter: where do solutions flow, where do they get trapped, where do they blow up, and where does existence/uniqueness fail just because the field isn’t even defined? That’s the perspective shift I wish every ODE course forces early and it’s exactly why I keep pairing math with animation. #DifferentialEquations #ODEs #VectorFields #MathAnimation #Mathematics

Mathelirium

53,841 views • 10 months ago

The first phrase of Beethoven's Ode to Joy is a single formula: d = ⌊2.27 + 1.2 sin(π(s + 1)/8) − cos(πs/4)⌋ with s = ⌊x⌋ + ½, played at f = 261.63 · 2ʸ Hz. The tune moves only by scale steps, so two harmonics carry its shape and the floor snaps each beat onto a degree: E E F G G F E D C C D E E D D. Then ⌊(12d + 5)/7⌋ = 0, 2, 4, 5, 7, 9, 11 turns degrees into semitones. The major scale itself is a floor: seven notes spread evenly over twelve.

The first phrase of Beethoven's Ode to Joy is a single formula: d = ⌊2.27 + 1.2 sin(π(s + 1)/8) − cos(πs/4)⌋ with s = ⌊x⌋ + ½, played at f = 261.63 · 2ʸ Hz. The tune moves only by scale steps, so two harmonics carry its shape and the floor snaps each beat onto a degree: E E F G G F E D C C D E E D D. Then ⌊(12d + 5)/7⌋ = 0, 2, 4, 5, 7, 9, 11 turns degrees into semitones. The major scale itself is a floor: seven notes spread evenly over twelve.

matharium

13,817 views • 1 month ago

A pen is driven by two perpendicular oscillations, x = 3 cos(50t) and y = 3 sin(49t). Since 49t = 50t − t, each lap of the x-wave is an ellipse x = 3 cos u, y = 3 sin(u − t) whose phase lags by the slowly growing t. It starts as a circle, flattens onto the diagonal y = −x at t = π/2, reopens at π, flattens onto y = x at 3π/2 and closes at 2π. Any two frequencies one apart weave the same square: n laps against n − 1, one full turn of phase.

A pen is driven by two perpendicular oscillations, x = 3 cos(50t) and y = 3 sin(49t). Since 49t = 50t − t, each lap of the x-wave is an ellipse x = 3 cos u, y = 3 sin(u − t) whose phase lags by the slowly growing t. It starts as a circle, flattens onto the diagonal y = −x at t = π/2, reopens at π, flattens onto y = x at 3π/2 and closes at 2π. Any two frequencies one apart weave the same square: n laps against n − 1, one full turn of phase.

matharium

12,779 views • 14 days ago

Backpropagation by hand ✍️ ~ 11 steps walkthrough below Backpropagation is the algorithm that actually trains a neural network, and it is where most people stop following along. It is not calculus you cannot do. It is matrix multiplication, working backward, one layer at a time. So I drew and calculated one entirely by hand. Goal: push the loss gradient back through a 3-layer network and land on a new value for every weight and bias. = 1. Given = A 3-layer perceptron, an input X, predictions Ypred = [0.5, 0.5, 0], and the truth Ytarget = [0, 1, 0]. = 2. Backprop gradient cells = Let us draw empty cells for every gradient we are about to compute. The shape of the answer comes first. = 3. Layer 3 softmax = We get dL/dz3 straight from Ypred minus Ytarget = [0.5, -0.5, 0]. No chain rule needed, and that shortcut is the whole reason softmax and cross-entropy are paired. = 4. Layer 3 weights and biases = Let us multiply dL/dz3 by [a2 | 1]. One multiplication gives the gradient for W3 and b3 together. = 5. Layer 2 activations = We multiply dL/dz3 by W3 to get dL/da2. The gradient moves back across a layer the same way the signal moved forward. = 6. Layer 2 ReLU = Let us pass it through the gate: keep the gradient where the activation was positive, zero it everywhere else. = 7. Layer 2 weights and biases = We multiply dL/dz2 by [a1 | 1]. The same figure as step 4, one layer up. = 8. Layer 1 activations = Let us multiply dL/dz2 by W2. = 9. Layer 1 ReLU = We apply the same gate again, now on a1. = 10. Layer 1 weights and biases = Let us multiply dL/dz1 by [x | 1], and every weight in the network now has a gradient. = 11. Update = We subtract, and the network has learned. In practice a learning rate scales this step. The gradients: dL/dz3 = [0.5, -0.5, 0] dL/da1 = [1, -2, 2, -1] dL/dz1 = [0, -2, 2, -1] The takeaway: matrix multiplication is all you need. Just like the forward pass, backpropagation is matrix multiplications end to end. You can do every one by hand, slowly and imperfectly, which is exactly why a GPU's ability to do them fast mattered so much to deep learning. 💾 Save this post!

Backpropagation by hand ✍️ ~ 11 steps walkthrough below Backpropagation is the algorithm that actually trains a neural network, and it is where most people stop following along. It is not calculus you cannot do. It is matrix multiplication, working backward, one layer at a time. So I drew and calculated one entirely by hand. Goal: push the loss gradient back through a 3-layer network and land on a new value for every weight and bias. = 1. Given = A 3-layer perceptron, an input X, predictions Ypred = [0.5, 0.5, 0], and the truth Ytarget = [0, 1, 0]. = 2. Backprop gradient cells = Let us draw empty cells for every gradient we are about to compute. The shape of the answer comes first. = 3. Layer 3 softmax = We get dL/dz3 straight from Ypred minus Ytarget = [0.5, -0.5, 0]. No chain rule needed, and that shortcut is the whole reason softmax and cross-entropy are paired. = 4. Layer 3 weights and biases = Let us multiply dL/dz3 by [a2 | 1]. One multiplication gives the gradient for W3 and b3 together. = 5. Layer 2 activations = We multiply dL/dz3 by W3 to get dL/da2. The gradient moves back across a layer the same way the signal moved forward. = 6. Layer 2 ReLU = Let us pass it through the gate: keep the gradient where the activation was positive, zero it everywhere else. = 7. Layer 2 weights and biases = We multiply dL/dz2 by [a1 | 1]. The same figure as step 4, one layer up. = 8. Layer 1 activations = Let us multiply dL/dz2 by W2. = 9. Layer 1 ReLU = We apply the same gate again, now on a1. = 10. Layer 1 weights and biases = Let us multiply dL/dz1 by [x | 1], and every weight in the network now has a gradient. = 11. Update = We subtract, and the network has learned. In practice a learning rate scales this step. The gradients: dL/dz3 = [0.5, -0.5, 0] dL/da1 = [1, -2, 2, -1] dL/dz1 = [0, -2, 2, -1] The takeaway: matrix multiplication is all you need. Just like the forward pass, backpropagation is matrix multiplications end to end. You can do every one by hand, slowly and imperfectly, which is exactly why a GPU's ability to do them fast mattered so much to deep learning. 💾 Save this post!

Tom Yeh

964,224 views • 3 months ago

Get out 🍴1 Get the f*ck out🍴2 Get the f*ck out 🍴3 Get the f*ck out 🍴4 Get out 5🍴thrown Good morning 😒

Get out 🍴1 Get the fck out🍴2 Get the fck out 🍴3 Get the f*ck out 🍴4 Get out 5🍴thrown Good morning 😒

Cool Girl

258,572 views • 1 year ago

At the right distance, you can hit f+1+2 after Clive's 10f punish 🙂

At the right distance, you can hit f+1+2 after Clive's 10f punish 🙂

QAD | Yagami

34,787 views • 1 year ago

Variational Autoencoder by hand ✍️ ~ 11 steps walkthrough below A VAE learns the structure of your data, the mean and variance of its hidden features, and then generates new data from that structure. A GAN only learns to fool a discriminator. It can make convincing fakes without ever knowing what the data is really made of. That is the difference, and it is the whole reason VAEs matter. In 2024 ICLR gave its first ever Test of Time Award to the VAE paper, "Auto-Encoding Variational Bayes" by Diederik Kingma and Max Welling, ten years on. How does it work? Goal: encode three inputs into a distribution, sample from it, decode it back, and read every loss gradient off the page. = 1. Given = Three training examples X1, X2, X3, copied to the bottom as their own targets. Reconstructing your own input is what puts the "auto", meaning self, in autoencoder. = 2. Encoder, layer 1 = Let us multiply the inputs by weights and biases, then apply ReLU, crossing out every negative. = 3. Mean and standard deviation = We multiply the features by two more weight sets. The first predicts the means μ of the latent distributions, the second their standard deviations σ. = 4. A random offset = Let us sample ε from a standard normal, mean 0 and variance 1, and multiply it by σ. This is a random step away from the mean, scaled by how uncertain each feature is. = 5. Mean plus offset = We add the offset back onto μ, and these become the decoder's inputs. Keeping the randomness out in ε is the reparameterization trick: it lets gradients flow straight through the sampling. = 6. Decoder, layer 1 = Let us multiply by weights and biases and apply ReLU again. Here -4 is crossed out. = 7. Decoder, layer 2 = We multiply once more. The output Y is the decoder's attempt to rebuild X from the sampled distribution. = 8. Gradient for the mean = Let us push μ toward 0. A lot of math, the SGVB estimator, collapses the KL gradient to simply μ itself. = 9. Gradient for the standard deviation = We want σ to approach 1. = 10. And its formula = That same math simplifies the gradient to σ minus 1/σ. = 11. Reconstruction gradient = We want the reconstruction Y to match the input X. Mean squared error simplifies its gradient to Y minus X. Takeaway: the two gradients you just calculated each sit at the heart of a modern method, so one VAE teaches you both. The KL divergence is the penalty RLHF like GRPO uses to keep a fine-tuned model from drifting off its base. The reconstruction loss, plain mean squared error, is exactly what trains a diffusion model to denoise. Draw one VAE by hand and you have quietly learned the core of both. 💾 Save this post!

Variational Autoencoder by hand ✍️ ~ 11 steps walkthrough below A VAE learns the structure of your data, the mean and variance of its hidden features, and then generates new data from that structure. A GAN only learns to fool a discriminator. It can make convincing fakes without ever knowing what the data is really made of. That is the difference, and it is the whole reason VAEs matter. In 2024 ICLR gave its first ever Test of Time Award to the VAE paper, "Auto-Encoding Variational Bayes" by Diederik Kingma and Max Welling, ten years on. How does it work? Goal: encode three inputs into a distribution, sample from it, decode it back, and read every loss gradient off the page. = 1. Given = Three training examples X1, X2, X3, copied to the bottom as their own targets. Reconstructing your own input is what puts the "auto", meaning self, in autoencoder. = 2. Encoder, layer 1 = Let us multiply the inputs by weights and biases, then apply ReLU, crossing out every negative. = 3. Mean and standard deviation = We multiply the features by two more weight sets. The first predicts the means μ of the latent distributions, the second their standard deviations σ. = 4. A random offset = Let us sample ε from a standard normal, mean 0 and variance 1, and multiply it by σ. This is a random step away from the mean, scaled by how uncertain each feature is. = 5. Mean plus offset = We add the offset back onto μ, and these become the decoder's inputs. Keeping the randomness out in ε is the reparameterization trick: it lets gradients flow straight through the sampling. = 6. Decoder, layer 1 = Let us multiply by weights and biases and apply ReLU again. Here -4 is crossed out. = 7. Decoder, layer 2 = We multiply once more. The output Y is the decoder's attempt to rebuild X from the sampled distribution. = 8. Gradient for the mean = Let us push μ toward 0. A lot of math, the SGVB estimator, collapses the KL gradient to simply μ itself. = 9. Gradient for the standard deviation = We want σ to approach 1. = 10. And its formula = That same math simplifies the gradient to σ minus 1/σ. = 11. Reconstruction gradient = We want the reconstruction Y to match the input X. Mean squared error simplifies its gradient to Y minus X. Takeaway: the two gradients you just calculated each sit at the heart of a modern method, so one VAE teaches you both. The KL divergence is the penalty RLHF like GRPO uses to keep a fine-tuned model from drifting off its base. The reconstruction loss, plain mean squared error, is exactly what trains a diffusion model to denoise. Draw one VAE by hand and you have quietly learned the core of both. 💾 Save this post!

Tom Yeh

17,011 views • 3 months ago

Global Strike Command unveiled its first demonstration tri-bomber formation at the Hyundai Air and Sea Show at Miami Beach, Florida, May 24, 2025. Memorial Day Weekend B-1, B-2, B-52, A-10, F-15, F-16, F-22

Global Strike Command unveiled its first demonstration tri-bomber formation at the Hyundai Air and Sea Show at Miami Beach, Florida, May 24, 2025. Memorial Day Weekend B-1, B-2, B-52, A-10, F-15, F-16, F-22

Turbine Traveller

132,983 views • 1 year ago