Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

➡️Machine Learning from Scratch, part 51: dropout. At every training step each hidden neuron flips its own coin and about half of them are switched off, a new random half every step. No neuron can lean on its neighbours, so the knowledge spreads over many neurons, and only patterns... that survive these small tests get stronger. After training, dropout is switched off and all neurons work together, like the average of many smaller networks that share the same weights. The code (dropout_reel.py, 16 lines of numpy) trains the same 1-64-1 tanh network twice on the same 12 noisy points of sin(3x), 60,000 steps of plain gradient descent, once the plain way and once with inverted dropout (chance one half, survivors multiplied by two). On 99 new points of the same curve the plain network ends at an error of 0.135 (its training error is almost zero, and its test error grows from 0.026 at step 100), the dropout network at 0.045: three times smaller, with a training error of 0.069. Sources: Hinton et al. 2012, arXiv:1207.0580; Srivastava et al. 2014, JMLR 15. #machinelearning #ai #coding #neuralnetworks #deeplearningshow more

MachineLearningToGo

27,900 subscribers

14,500 views • 4 days ago •via TikTok

Science & Technology #machinelearning #ai #coding #neuralnetworks #deeplearning

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

➡️ Part 9 of learning ML code from scratch Six weights, one wrong answer, and not one of them knows it was at fault. Backpropagation is how they find out. Two questions guide the video: what even is backpropagation, and why do models need it? Step 1, the forward pass. Two numbers go into a tiny network, run through a hidden layer, and one number comes out. Ours says 0.700. We wanted 1.0, so square the difference and the loss is 0.09. That single number is how wrong the model is. Step 2, the question. Six weights produced that answer together. Which of them is to blame, and by how much? Step 3, the one idea. Change a single weight by a tiny amount and watch the error move. How much the error changes per unit of weight is the gradient of that weight. In the video you can see it happen: the weight wobbles and the error bar answers. Step 4, the shortcut. Every weight running into the same neuron starts from one shared number, the blame of that neuron. Multiply that blame by the value each weight carried and you have its gradient. One blame, computed once, and every weight into that neuron gets its own gradient from it. That is why backpropagation is fast enough to train anything at all. Step 5, backwards. The blame starts at the output and every layer hands its share to the layer in front of it, back through the same network the data came forward through. It goes layer by layer, and inside one layer every neuron is done together. Step 6, the update. The optimizer moves every weight against its own gradient, the whole weight matrix in one go, so no weight waits for its turn. All six gradients here are negative, so all six weights go up. After one step the answer reads 0.769. After sixty it reads 0.968 and the error is down to 0.001. The honest part: every gradient in the video was checked against the measured change of the error. Move that one weight, see what the error does, divide. Backprop and the measurement agree to nine decimal places, otherwise nothing would have rendered. #machinelearning #backpropagation #neuralnetworks #python #coding

➡️ Part 9 of learning ML code from scratch Six weights, one wrong answer, and not one of them knows it was at fault. Backpropagation is how they find out. Two questions guide the video: what even is backpropagation, and why do models need it? Step 1, the forward pass. Two numbers go into a tiny network, run through a hidden layer, and one number comes out. Ours says 0.700. We wanted 1.0, so square the difference and the loss is 0.09. That single number is how wrong the model is. Step 2, the question. Six weights produced that answer together. Which of them is to blame, and by how much? Step 3, the one idea. Change a single weight by a tiny amount and watch the error move. How much the error changes per unit of weight is the gradient of that weight. In the video you can see it happen: the weight wobbles and the error bar answers. Step 4, the shortcut. Every weight running into the same neuron starts from one shared number, the blame of that neuron. Multiply that blame by the value each weight carried and you have its gradient. One blame, computed once, and every weight into that neuron gets its own gradient from it. That is why backpropagation is fast enough to train anything at all. Step 5, backwards. The blame starts at the output and every layer hands its share to the layer in front of it, back through the same network the data came forward through. It goes layer by layer, and inside one layer every neuron is done together. Step 6, the update. The optimizer moves every weight against its own gradient, the whole weight matrix in one go, so no weight waits for its turn. All six gradients here are negative, so all six weights go up. After one step the answer reads 0.769. After sixty it reads 0.968 and the error is down to 0.001. The honest part: every gradient in the video was checked against the measured change of the error. Move that one weight, see what the error does, divide. Backprop and the measurement agree to nine decimal places, otherwise nothing would have rendered. #machinelearning #backpropagation #neuralnetworks #python #coding

MachineLearningToGo

210,300 views • 1 month ago

Geoffrey Hinton once noticed that the tellers at his bank kept changing. When he asked one of them why, the teller didn't know, only that they got moved around a lot. Hinton guessed the reason... if staff never work with the same people for long, it's much harder for a few of them to team up and cheat the bank. That thought became one of the most useful tricks in deep learning. While a neural network trains, you randomly switch off about half its neurons at every step. No neuron can lean on a particular partner, so each one has to learn something useful on its own. The trick is called dropout, and it helped AlexNet win the 2012 ImageNet contest, which set off the modern AI boom. It's still built into many of today's networks.

Geoffrey Hinton once noticed that the tellers at his bank kept changing. When he asked one of them why, the teller didn't know, only that they got moved around a lot. Hinton guessed the reason... if staff never work with the same people for long, it's much harder for a few of them to team up and cheat the bank. That thought became one of the most useful tricks in deep learning. While a neural network trains, you randomly switch off about half its neurons at every step. No neuron can lean on a particular partner, so each one has to learn something useful on its own. The trick is called dropout, and it helped AlexNet win the 2012 ImageNet contest, which set off the modern AI boom. It's still built into many of today's networks.

Mathelirium

274,666 views • 18 days ago

➡️ Machine Learning from Scratch, part 47: the Hopfield network. In 1982 the physicist John Hopfield described a network that works as a memory, and 42 years later this work was part of the Nobel Prize in Physics. Each neuron is one pixel that is either on or off, and every neuron is connected to every other neuron, so 100 pixels need 4,950 connections. There are no layers: signals go back and forth until the network settles. Each neuron takes a weighted sum of its inputs and switches on if the sum is positive, off if it is negative. Storing a picture is simple: for every pair of pixels with the same colour the connection gets stronger, otherwise weaker. Every stored picture becomes a valley in the energy landscape, and no switch can ever raise the energy, so a noisy picture rolls downhill into the nearest stored memory. The code (hopfield.py, 16 lines of numpy, pictures in pics.txt) stores three 10 x 10 pictures, a heart, a house and a smiley, and flips 30 of the 100 pixels in each. After a single pass over all neurons all three come back exactly; the energy of the heart falls from -6.90 to -48.90, the energy of the stored heart. With 100 neurons and random patterns (10 flipped pixels, 20 runs) it recalls 5 patterns every time, 10 patterns in 94.5 % of the cases, 14 in 68 %, 20 in about one in four (24 %) and 30 in 1 %. Honest notes: the weights are kept as whole numbers (100 times the Hebb weight), so a sum of exactly zero switches the neuron on, as the rule says. The landscape in the video is a picture of the idea; the valley floors and the height of the rolling ball are the real energies from the run. #machinelearning #ai #coding #neuralnetworks #nobelprize

➡️ Machine Learning from Scratch, part 47: the Hopfield network. In 1982 the physicist John Hopfield described a network that works as a memory, and 42 years later this work was part of the Nobel Prize in Physics. Each neuron is one pixel that is either on or off, and every neuron is connected to every other neuron, so 100 pixels need 4,950 connections. There are no layers: signals go back and forth until the network settles. Each neuron takes a weighted sum of its inputs and switches on if the sum is positive, off if it is negative. Storing a picture is simple: for every pair of pixels with the same colour the connection gets stronger, otherwise weaker. Every stored picture becomes a valley in the energy landscape, and no switch can ever raise the energy, so a noisy picture rolls downhill into the nearest stored memory. The code (hopfield.py, 16 lines of numpy, pictures in pics.txt) stores three 10 x 10 pictures, a heart, a house and a smiley, and flips 30 of the 100 pixels in each. After a single pass over all neurons all three come back exactly; the energy of the heart falls from -6.90 to -48.90, the energy of the stored heart. With 100 neurons and random patterns (10 flipped pixels, 20 runs) it recalls 5 patterns every time, 10 patterns in 94.5 % of the cases, 14 in 68 %, 20 in about one in four (24 %) and 30 in 1 %. Honest notes: the weights are kept as whole numbers (100 times the Hebb weight), so a sum of exactly zero switches the neuron on, as the rule says. The landscape in the video is a picture of the idea; the valley floors and the height of the rolling ball are the real energies from the run. #machinelearning #ai #coding #neuralnetworks #nobelprize

MachineLearningToGo

11,900 views • 12 days ago

Dropout by hand ✍️ ~ 10 steps walkthrough below Dropout is the simplest trick in deep learning that actually works: during training you randomly switch neurons off, so the network cannot lean on any one of them. It is two lines of code and almost nobody has worked through what those lines do to the numbers. So I drew and calculated one entirely by hand. Goal: train one pass through a small network with two dropout layers, then run inference with dropout switched off. The network: Linear(2,4), ReLU, Dropout(0.5), Linear(4,3), ReLU, Dropout(0.33), Linear(3,2). = 1. Given = A training set of two examples, X1 and X2, and the weight matrices for all three linear layers. = 2. Draw the first random numbers = Let us draw 4 random numbers, one per neuron in the first hidden layer. Above 0.5 we keep (◯), below we drop (╳). Here that gives [◯, ╳, ◯, ╳]. = 3. Build the first dropout matrix = We turn that pattern into a diagonal matrix. The scaling factor is 1/(1-p) = 2, so a kept neuron gets 2 and a dropped one gets 0. Multiplying by it does both jobs at once: it deletes the 2nd and 4th neurons and doubles the two that survive. = 4. Draw the second random numbers = Let us do it again for the 3 neurons in the next layer, this time against p = 0.33. The result is [◯, ◯, ╳]. = 5. Build the second dropout matrix = We set the diagonal to 1.5 where kept and 0 where dropped. Only the 3rd neuron goes. = 6. Feed forward = Let us run the whole thing top to bottom: one matrix multiplication per layer, ReLU setting the negatives to zero, and the two dropout matrices doing their work in between. The outputs Y come out at the bottom. = 7. MSE loss gradients = We compare Y against the targets Y', subtract, and multiply each element by 2. That is the whole gradient of the mean squared error. = 8. Update the weights = Let us push those gradients back through the network and update the weights (marked in light red). = 9. Deactivate dropout = Training is over, so we set both dropout matrices to the identity. Every neuron is back, and nothing is scaled. = 10. Feed forward again = One more pass, this time on unseen data, to make the prediction. You have just trained and run a network with dropout by hand. ✍️ The outputs: Training outputs Y = [-6, 9; 13, 4] Loss gradients = [-4, 4; 6, -2] Inference outputs = [13, 13; 4, 3] 💾 Save this post! #AIbyHand #Dropout #DeepLearning #NeuralNetworks

Dropout by hand ✍️ ~ 10 steps walkthrough below Dropout is the simplest trick in deep learning that actually works: during training you randomly switch neurons off, so the network cannot lean on any one of them. It is two lines of code and almost nobody has worked through what those lines do to the numbers. So I drew and calculated one entirely by hand. Goal: train one pass through a small network with two dropout layers, then run inference with dropout switched off. The network: Linear(2,4), ReLU, Dropout(0.5), Linear(4,3), ReLU, Dropout(0.33), Linear(3,2). = 1. Given = A training set of two examples, X1 and X2, and the weight matrices for all three linear layers. = 2. Draw the first random numbers = Let us draw 4 random numbers, one per neuron in the first hidden layer. Above 0.5 we keep (◯), below we drop (╳). Here that gives [◯, ╳, ◯, ╳]. = 3. Build the first dropout matrix = We turn that pattern into a diagonal matrix. The scaling factor is 1/(1-p) = 2, so a kept neuron gets 2 and a dropped one gets 0. Multiplying by it does both jobs at once: it deletes the 2nd and 4th neurons and doubles the two that survive. = 4. Draw the second random numbers = Let us do it again for the 3 neurons in the next layer, this time against p = 0.33. The result is [◯, ◯, ╳]. = 5. Build the second dropout matrix = We set the diagonal to 1.5 where kept and 0 where dropped. Only the 3rd neuron goes. = 6. Feed forward = Let us run the whole thing top to bottom: one matrix multiplication per layer, ReLU setting the negatives to zero, and the two dropout matrices doing their work in between. The outputs Y come out at the bottom. = 7. MSE loss gradients = We compare Y against the targets Y', subtract, and multiply each element by 2. That is the whole gradient of the mean squared error. = 8. Update the weights = Let us push those gradients back through the network and update the weights (marked in light red). = 9. Deactivate dropout = Training is over, so we set both dropout matrices to the identity. Every neuron is back, and nothing is scaled. = 10. Feed forward again = One more pass, this time on unseen data, to make the prediction. You have just trained and run a network with dropout by hand. ✍️ The outputs: Training outputs Y = [-6, 9; 13, 4] Loss gradients = [-4, 4; 6, -2] Inference outputs = [13, 13; 4, 3] 💾 Save this post! #AIbyHand #Dropout #DeepLearning #NeuralNetworks

Tom Yeh

14,442 views • 3 months ago

➡️ Machine Learning from Scratch, part 40: is linear regression machine learning? Linear regression is machine learning, because it learns a weight and a bias from examples with known answers, which makes it supervised learning in its simplest form. What makes it special is that the best weight and bias can also be computed in one step with the normal equation, w = (X^T X)^-1 X^T y, so standard tools usually solve it directly with a least squares routine without any training loop. Gradient descent comes back when the data gets really huge, and it trains neural networks, where there is generally no such formula. We ran everything ourselves and downloaded nothing. linreg_ml.py (14 lines of numpy) takes the same eight points as part 1. Gradient descent with the same settings as part 1 (start at 0, learning rate 0.01) still has a bias of 1.995 after 1,000 steps, while the formula gives 2.029 (weight 0.796), and it needs 2,052 steps to come within 0.0005 of the formula. The least squares routine gives the same numbers as the formula. Honest notes: on eight points the formula is instant, and on millions of rows with many features the picture changes, which is when step by step training pays off. #machinelearning #ai #linearregression #datascience #coding

➡️ Machine Learning from Scratch, part 40: is linear regression machine learning? Linear regression is machine learning, because it learns a weight and a bias from examples with known answers, which makes it supervised learning in its simplest form. What makes it special is that the best weight and bias can also be computed in one step with the normal equation, w = (X^T X)^-1 X^T y, so standard tools usually solve it directly with a least squares routine without any training loop. Gradient descent comes back when the data gets really huge, and it trains neural networks, where there is generally no such formula. We ran everything ourselves and downloaded nothing. linreg_ml.py (14 lines of numpy) takes the same eight points as part 1. Gradient descent with the same settings as part 1 (start at 0, learning rate 0.01) still has a bias of 1.995 after 1,000 steps, while the formula gives 2.029 (weight 0.796), and it needs 2,052 steps to come within 0.0005 of the formula. The least squares routine gives the same numbers as the formula. Honest notes: on eight points the formula is instant, and on millions of rows with many features the picture changes, which is when step by step training pays off. #machinelearning #ai #linearregression #datascience #coding

MachineLearningToGo

22,700 views • 23 days ago

➡️ ML news part 1 An AI just fixed another AI by rewriting its weights directly, without any training. Machine Learning News, part 1: weight space programming. What is new, and why it matters. Until now, the weights of a neural network were set by training, and reading them took special tools built for that one job. A new report from researchers at MIT CSAIL shows that a general AI model, here GPT-6 Astra used as it is, can read and write those weights directly as plain lists of numbers. Test 1, reading. The model was given only the 17,253 weights of four small game playing networks (Breakout, Pong, Pac-Man, Freeway) and named the right game for all four. Test 2, writing. Without any data, loss or gradients, it wrote networks for three MinAtar games, and all three beat networks trained with PPO for 100 million steps. In Breakout it scored 517.4 points, against 25.3 after 500 million training steps. Test 3, repairing. The small open language model Qwen3.5-0.8B ran into its length limit on 93 percent of MATH-500 answers because it got stuck in loops. The model traced the loop to one direction written by the MLPs of layers 15 and 19 and removed it. That changed 131,072 weights, which is 0.016 percent of the model. MATH-500 went from 7.9 to 48.1 percent, IFEval from 21.0 to 62.3 percent, GSM8K from 13.8 to 67.1 percent, all with no training. The edit was chosen with a budget of 50 trial runs on practice problems, and the test sets were never seen. To be fair, the authors say the fix removed an obstacle and added no new ability, and on another small model it brought no improvement. This is an early report and not yet peer reviewed. Source: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), "Sparks of Weight-Space Programming", blog post, November 2, 2026, (early report, not peer reviewed). Screenshots from the report are shown as quotes. Model names are given only to name the source. #machinelearning #ai #claude #neuralnetworks #llm

➡️ ML news part 1 An AI just fixed another AI by rewriting its weights directly, without any training. Machine Learning News, part 1: weight space programming. What is new, and why it matters. Until now, the weights of a neural network were set by training, and reading them took special tools built for that one job. A new report from researchers at MIT CSAIL shows that a general AI model, here GPT-6 Astra used as it is, can read and write those weights directly as plain lists of numbers. Test 1, reading. The model was given only the 17,253 weights of four small game playing networks (Breakout, Pong, Pac-Man, Freeway) and named the right game for all four. Test 2, writing. Without any data, loss or gradients, it wrote networks for three MinAtar games, and all three beat networks trained with PPO for 100 million steps. In Breakout it scored 517.4 points, against 25.3 after 500 million training steps. Test 3, repairing. The small open language model Qwen3.5-0.8B ran into its length limit on 93 percent of MATH-500 answers because it got stuck in loops. The model traced the loop to one direction written by the MLPs of layers 15 and 19 and removed it. That changed 131,072 weights, which is 0.016 percent of the model. MATH-500 went from 7.9 to 48.1 percent, IFEval from 21.0 to 62.3 percent, GSM8K from 13.8 to 67.1 percent, all with no training. The edit was chosen with a budget of 50 trial runs on practice problems, and the test sets were never seen. To be fair, the authors say the fix removed an obstacle and added no new ability, and on another small model it brought no improvement. This is an early report and not yet peer reviewed. Source: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), "Sparks of Weight-Space Programming", blog post, November 2, 2026, (early report, not peer reviewed). Screenshots from the report are shown as quotes. Model names are given only to name the source. #machinelearning #ai #claude #neuralnetworks #llm

MachineLearningToGo

55,800 views • 9 days ago

The Moore curve of order 4 is a loop of 256 unit steps through every cell of a 16 × 16 grid. Each coordinate has its own real Fourier series, and one chain of arms per axis draws it. The second half of the loop is the first half turned 180°, so every even harmonic is zero, and its mirror leaves x only sines and y only cosines. A curve that wiggles everywhere converges slowly: 32.3 % of the height at N = 1, still 0.47 % at N = 255.

The Moore curve of order 4 is a loop of 256 unit steps through every cell of a 16 × 16 grid. Each coordinate has its own real Fourier series, and one chain of arms per axis draws it. The second half of the loop is the first half turned 180°, so every even harmonic is zero, and its mirror leaves x only sines and y only cosines. A curve that wiggles everywhere converges slowly: 32.3 % of the height at N = 1, still 0.47 % at N = 255.

matharium

72,548 views • 1 month ago

A Neural Network Can Grow New Neurons Where It Is Confused? In 1994, Bernd Fritzke published A Growing Neural Gas Network Learns Topologies. He introduced a network that starts small, follows incoming data, and inserts new neurons where its error is highest. In the animation, the fog is the drifting data. The glowing nodes are neurons. The fibers are learned connections. The network grows into a living skeleton of the manifold.

A Neural Network Can Grow New Neurons Where It Is Confused? In 1994, Bernd Fritzke published A Growing Neural Gas Network Learns Topologies. He introduced a network that starts small, follows incoming data, and inserts new neurons where its error is highest. In the animation, the fog is the drifting data. The glowing nodes are neurons. The fibers are learned connections. The network grows into a living skeleton of the manifold.

Mathelirium

38,431 views • 6 months ago

➡️ Machine Learning from Scratch, part 31: the transformer. A transformer turns every token into a list of numbers, adds its position, and then lets every token look at the others. That step is called attention: each token has a query (what am I looking for), a key (what do I contain) and a value (what I pass on). The better a query matches a key, the more of that value flows in. Real transformers repeat this in many layers with several attention heads each. The design comes from the 2017 Google paper "Attention Is All You Need" by Ashish Vaswani and colleagues, first used for translation. A large language model is a transformer trained to predict the next token, again and again, on a huge amount of text. Our own run, nothing downloaded. attention.py is one self-attention layer in 16 lines of numpy. Every word gets a query, a key and a value of 8 numbers, starting random, trained from scratch with hand-written backpropagation on 192 made-up sentences like "the dog ate the carrot because it was hungry". The task: find out which pet "it" means. On 64 new test sentences "it" puts 99.95 percent of its attention on the right word on average (the lowest single sentence gets 99.9 percent; the program prints this as 100%). Before training it was 10 percent, spread evenly. The cat sentence from the video is one of the test sentences. Honest notes: our task is easy on purpose. Every sentence has exactly one pet, so "it" only has to learn to look for pets, and it learns that within the first pass. Real sentences with two candidates need more context, which is what many layers are for. Our small version has no positions, no scaling, no mask, one head, and only "it" asks a question. Every word of the test sentences also appears in training; the combinations are new. The 3D space in the video shows the real keys of our model, projected to three axes (two of them stretched so you can see them). The stacked layers are a sketch. #machinelearning #ai #transformer #python #llm

➡️ Machine Learning from Scratch, part 31: the transformer. A transformer turns every token into a list of numbers, adds its position, and then lets every token look at the others. That step is called attention: each token has a query (what am I looking for), a key (what do I contain) and a value (what I pass on). The better a query matches a key, the more of that value flows in. Real transformers repeat this in many layers with several attention heads each. The design comes from the 2017 Google paper "Attention Is All You Need" by Ashish Vaswani and colleagues, first used for translation. A large language model is a transformer trained to predict the next token, again and again, on a huge amount of text. Our own run, nothing downloaded. attention.py is one self-attention layer in 16 lines of numpy. Every word gets a query, a key and a value of 8 numbers, starting random, trained from scratch with hand-written backpropagation on 192 made-up sentences like "the dog ate the carrot because it was hungry". The task: find out which pet "it" means. On 64 new test sentences "it" puts 99.95 percent of its attention on the right word on average (the lowest single sentence gets 99.9 percent; the program prints this as 100%). Before training it was 10 percent, spread evenly. The cat sentence from the video is one of the test sentences. Honest notes: our task is easy on purpose. Every sentence has exactly one pet, so "it" only has to learn to look for pets, and it learns that within the first pass. Real sentences with two candidates need more context, which is what many layers are for. Our small version has no positions, no scaling, no mask, one head, and only "it" asks a question. Every word of the test sentences also appears in training; the combinations are new. The 3D space in the video shows the real keys of our model, projected to three axes (two of them stretched so you can see them). The stacked layers are a sketch. #machinelearning #ai #transformer #python #llm

MachineLearningToGo

79,900 views • 1 month ago

Most coins trade against SOL. The coins launched here trade against another coin, and feed on it. Pick any memecoin with at least 20 SOL of real liquidity as the host and launch your parasite on it. From then on, the only way in or out of your coin goes through the host. There is no SOL pool. Every trade pays its 1% fee in the host coin. The program claims that fee and burns half of what it collects, in the same instruction. The busier the parasite, the smaller the host's supply. And a parasite can carry parasites of its own, three levels deep, each one eating the coin beneath it. Website: CA: 3kmygWKZBkCYrgZHKfiuB9UFKTcDLTFFsKo3BWpmpump $PARASITE

Most coins trade against SOL. The coins launched here trade against another coin, and feed on it. Pick any memecoin with at least 20 SOL of real liquidity as the host and launch your parasite on it. From then on, the only way in or out of your coin goes through the host. There is no SOL pool. Every trade pays its 1% fee in the host coin. The program claims that fee and burns half of what it collects, in the same instruction. The busier the parasite, the smaller the host's supply. And a parasite can carry parasites of its own, three levels deep, each one eating the coin beneath it. Website: CA: 3kmygWKZBkCYrgZHKfiuB9UFKTcDLTFFsKo3BWpmpump $PARASITE

Parasite

88,534 views • 1 month ago

A Neural Sheet Discovers the Shape of Data A self-organizing map (SOM) starts as a flat sheet of neurons with no knowledge of the data around it. For every input sample, it finds the best-matching neuron, c = argminᵢ ‖x - wᵢ‖, and updates that neuron together with its neighbours, wᵢ ← wᵢ + η(t)hᶜᵢ(t)(x - wᵢ). As the neighbourhood radius shrinks during training, the sheet gradually bends and folds onto the hidden geometry of the dataset while preserving its neighbourhood structure. Developed by Finnish computer scientist Teuvo Kohonen, the Self-Organizing Map remains one of the most elegant examples of competitive learning, where order emerges from thousands of simple local updates. #MachineLearning #ArtificialIntelligence #SelfOrganizingMap #NeuralNetworks #DataVisualization #Mathematics #ComputerScience #Finland #TeuvoKohonen

A Neural Sheet Discovers the Shape of Data A self-organizing map (SOM) starts as a flat sheet of neurons with no knowledge of the data around it. For every input sample, it finds the best-matching neuron, c = argminᵢ ‖x - wᵢ‖, and updates that neuron together with its neighbours, wᵢ ← wᵢ + η(t)hᶜᵢ(t)(x - wᵢ). As the neighbourhood radius shrinks during training, the sheet gradually bends and folds onto the hidden geometry of the dataset while preserving its neighbourhood structure. Developed by Finnish computer scientist Teuvo Kohonen, the Self-Organizing Map remains one of the most elegant examples of competitive learning, where order emerges from thousands of simple local updates. #MachineLearning #ArtificialIntelligence #SelfOrganizingMap #NeuralNetworks #DataVisualization #Mathematics #ComputerScience #Finland #TeuvoKohonen

Mathelirium

177,699 views • 4 months ago

Your brain doesn't search through memories — it falls downhill on an energy landscape and lands in the nearest valley. This is the Hopfield network from 1982, and the same math now lives inside every transformer: the attention mechanism in modern LLMs is mathematically equivalent to a Hopfield network with exponential storage capacity. The idea: neurons are +1 or -1, connected by symmetric weights. Memories are local minima sculpted by Hebbian learning — neurons that fire together, wire together. Give the network a noisy input, and it descends toward the nearest stored memory one GPU-style majority vote at a time. Vanilla Hopfield nets cap at ~14% of neuron count as usable memories, but the modern extension blows past that. 23 minutes. Bookmark & watch today.

Your brain doesn't search through memories — it falls downhill on an energy landscape and lands in the nearest valley. This is the Hopfield network from 1982, and the same math now lives inside every transformer: the attention mechanism in modern LLMs is mathematically equivalent to a Hopfield network with exponential storage capacity. The idea: neurons are +1 or -1, connected by symmetric weights. Memories are local minima sculpted by Hebbian learning — neurons that fire together, wire together. Give the network a noisy input, and it descends toward the nearest stored memory one GPU-style majority vote at a time. Vanilla Hopfield nets cap at ~14% of neuron count as usable memories, but the modern extension blows past that. 23 minutes. Bookmark & watch today.

Eron

23,025 views • 1 month ago

Google's first rule for machine learning: don't use machine learning. A 3-line PID equation beats a neural network at balancing a robot. Proportional error, integral of accumulated error, derivative of error change rate - three terms, hand-tuned gains, no training data. The same control theory running inside every GPU cooling system and every server rack thermal regulator at every LLM data center on the planet. The difference shows up when physics change. Add a load, tilt the surface, face an unknown environment - the PID's hand-tuned K values break because they're locked to one set of physics. A neural network with ReLU activation functions captures nonlinear dynamics: a 10-degree lean with backward acceleration doesn't need the same response as a 10-degree lean standing still. Without activation functions a neural network is just y = mx + b. One bend in the function is what separates a linear controller from a system that adapts to anything. 8 minutes. Bookmark & watch today. The 3-line equation that beats AI until the world stops being predictable.

Google's first rule for machine learning: don't use machine learning. A 3-line PID equation beats a neural network at balancing a robot. Proportional error, integral of accumulated error, derivative of error change rate - three terms, hand-tuned gains, no training data. The same control theory running inside every GPU cooling system and every server rack thermal regulator at every LLM data center on the planet. The difference shows up when physics change. Add a load, tilt the surface, face an unknown environment - the PID's hand-tuned K values break because they're locked to one set of physics. A neural network with ReLU activation functions captures nonlinear dynamics: a 10-degree lean with backward acceleration doesn't need the same response as a 10-degree lean standing still. Without activation functions a neural network is just y = mx + b. One bend in the function is what separates a linear controller from a system that adapts to anything. 8 minutes. Bookmark & watch today. The 3-line equation that beats AI until the world stops being predictable.

Eron

53,187 views • 13 days ago

SOMEONE REPLAYED EVERY SINGLE PERSON WHO PLAYED THEIR VIBE CODED RACING GAME, ALL AT THE SAME TIME, ON THE SAME TRACK its a daily racing game, everyone gets the same track each day and tries to beat it. so he took every run from yesterday and played them back simultaneously as ghost cars > all 124 runs from that day racing at once, stacked on top of each other > you can see exactly where everyone takes the same line, and where people fall into a ditch > the pack splits at the same corners every time, so you can literally watch the track teaching people > every run gets stored, so the replay is just his own player data played back at once > a brand new track gets generated every single day, so its never the same race twice > the whole thing runs in the browser on three.js, no download, and he vibe coded the game, the graphics and the replay system himself its not a leaderboard telling you someone was faster, its 124 people making 124 different decisions at the exact same corner, at the same time, in front of you every daily game should do this. the runs are already there, you just have to show them all at once

Om Patel

73,298 views • 3 months ago

➡️ Quantitative Finance from Scratch, part 45: Statistical Arbitrage. The arbitrage from the last part was a profit without any risk. Statistical arbitrage is the practical version: many small bets that can each lose, but win on average. One of the best known and simplest forms is pairs trading: two stocks in the same business that move together. The gap between them, the spread = P_A − β · P_B (P_A price of stock A, P_B price of stock B, β the hedge ratio), keeps coming back to its average. That is the edge: the trade treats this return almost like a law, so the further the spread moves away, the more it should come back, and that way back is the profit. The rule: z = (spread − average) / standard deviation. When z reaches +2 or −2, you sell the expensive stock, buy the cheap one, and close both when the gap is back at its average. Because one stock is long and the other short, a move of the whole market mostly cancels out. News about one company can still hit a single pair, so funds make many such trades at once, and the surprises average out. The big risks: the link between two stocks can break for good, for example after a merger, and when many funds hold the same trades, they can all lose at once. The code (statarb_reel.py, 16 lines of plain Python without packages) is general on purpose: the spread from two prices and the hedge ratio, the z score, and the signal that sells the expensive stock and buys the cheap one at two standard deviations and closes at the average. If you want to go deeper, the link to my website is in my bio. #quantfinance #statisticalarbitrage #pairstrading #meanreversion #python Education only, not financial advice.

➡️ Quantitative Finance from Scratch, part 45: Statistical Arbitrage. The arbitrage from the last part was a profit without any risk. Statistical arbitrage is the practical version: many small bets that can each lose, but win on average. One of the best known and simplest forms is pairs trading: two stocks in the same business that move together. The gap between them, the spread = P_A − β · P_B (P_A price of stock A, P_B price of stock B, β the hedge ratio), keeps coming back to its average. That is the edge: the trade treats this return almost like a law, so the further the spread moves away, the more it should come back, and that way back is the profit. The rule: z = (spread − average) / standard deviation. When z reaches +2 or −2, you sell the expensive stock, buy the cheap one, and close both when the gap is back at its average. Because one stock is long and the other short, a move of the whole market mostly cancels out. News about one company can still hit a single pair, so funds make many such trades at once, and the surprises average out. The big risks: the link between two stocks can break for good, for example after a merger, and when many funds hold the same trades, they can all lose at once. The code (statarb_reel.py, 16 lines of plain Python without packages) is general on purpose: the spread from two prices and the hedge ratio, the z score, and the signal that sells the expensive stock and buys the cheap one at two standard deviations and closes at the average. If you want to go deeper, the link to my website is in my bio. #quantfinance #statisticalarbitrage #pairstrading #meanreversion #python Education only, not financial advice.

QuantFinanceToGo

37,500 views • 3 days ago

The outline of π, sampled at 2048 points, is a sum of circles: z(t) = ∑ cₖ eⁱᵏᵗ, the cₖ from its discrete Fourier transform. Circle k has radius |cₖ| and turns k times per lap. With the N largest, the widest gap to the outline is 41.6 % of the letter's height at N = 1, 1.37 % at 64, 0.36 % at 256. Finitely many circles make a smooth curve, and π has four sharp corners, so no chain is exact. The error stays below the sum of the radii left out.

The outline of π, sampled at 2048 points, is a sum of circles: z(t) = ∑ cₖ eⁱᵏᵗ, the cₖ from its discrete Fourier transform. Circle k has radius |cₖ| and turns k times per lap. With the N largest, the widest gap to the outline is 41.6 % of the letter's height at N = 1, 1.37 % at 64, 0.36 % at 256. Finitely many circles make a smooth curve, and π has four sharp corners, so no chain is exact. The error stays below the sum of the radii left out.

matharium

51,981 views • 1 month ago

Machine Learning from Scratch, part 1. Part 2 adds three lines to this exact program and the line turns into a neuron. #machinelearning #python #gradientdescent #linearregression #neuralnetworks

Machine Learning from Scratch, part 1. Part 2 adds three lines to this exact program and the line turns into a neuron. #machinelearning #python #gradientdescent #linearregression #neuralnetworks

MachineLearningToGo

357,300 views • 1 month ago

We often hear that machine learning models "learn patterns in data". But what does that actually look like in geometry? If you dropped a little elastic mesh into a cloud of points and let it learn, how would it fold itself to match the shape of the data? In this scene we watch a self-organizing map...a simple unsupervised neural model...learn the shape of a two-dimensional dataset arranged in a spiral arm. On top of this, we lay down a square grid of neurons whose weights live in the same plane. At the start, this grid is just a flat net floating across the cloud...it knows nothing about the structure underneath. Learning is a repeated game: pick a random data point, find the neuron whose weight is closest, and then nudge that neuron and its neighbours toward the point. Do this again and again, while slowly shrinking how far the neighbourhood influence spreads. #MachineLearning #ManifoldLearning #UnsupervisedLearning #NeuralMaps #GeometricML

We often hear that machine learning models "learn patterns in data". But what does that actually look like in geometry? If you dropped a little elastic mesh into a cloud of points and let it learn, how would it fold itself to match the shape of the data? In this scene we watch a self-organizing map...a simple unsupervised neural model...learn the shape of a two-dimensional dataset arranged in a spiral arm. On top of this, we lay down a square grid of neurons whose weights live in the same plane. At the start, this grid is just a flat net floating across the cloud...it knows nothing about the structure underneath. Learning is a repeated game: pick a random data point, find the neuron whose weight is closest, and then nudge that neuron and its neighbours toward the point. Do this again and again, while slowly shrinking how far the neighbourhood influence spreads. #MachineLearning #ManifoldLearning #UnsupervisedLearning #NeuralMaps #GeometricML

Mathelirium

139,930 views • 11 months ago

Steve Kerr on Steph Curry’s work ethic - “Just the consistency of his routine its like a metronome everyday its the exact same. He’s in the training room, he’s in the weight room, he’s on the court and its clock work, but there’s a sense of joy of energy within that work.

Steve Kerr on Steph Curry’s work ethic - “Just the consistency of his routine its like a metronome everyday its the exact same. He’s in the training room, he’s in the weight room, he’s on the court and its clock work, but there’s a sense of joy of energy within that work.

Ryan Pannone

86,126 views • 2 years ago

Backpropagation in Machine Learning In 1970, a Finnish master's student named Seppo Linnainmaa wrote a thesis on tracing a computer's tiny rounding errors backwards through a long calculation, one step at a time. Sixteen years later, David Rumelhart, Geoffrey Hinton and Ronald Williams showed that the same backward trick lets a Neural Network work out how to fix every one of its connections at once, and the world called it 'Backpropagation'. Every big AI model today still learns by passing its mistakes backwards in exactly this way.

Backpropagation in Machine Learning In 1970, a Finnish master's student named Seppo Linnainmaa wrote a thesis on tracing a computer's tiny rounding errors backwards through a long calculation, one step at a time. Sixteen years later, David Rumelhart, Geoffrey Hinton and Ronald Williams showed that the same backward trick lets a Neural Network work out how to fix every one of its connections at once, and the world called it 'Backpropagation'. Every big AI model today still learns by passing its mistakes backwards in exactly this way.

Mathelirium

13,116 views • 15 days ago