Skip navigation
➡️ Machine Learning from Scratch, part 44: overfitting detection A model overfits when it learns its…
0:00 / 1:21

➡️ Machine Learning from Scratch, part 44: overfitting detection A model overfits when it learns its…

66,700 views #machinelearning #ai #overfitting

➡️ Machine Learning from Scratch, part 44: overfitting detection A model overfits when it learns its training data by heart, noise included, and misses the pattern behind it. To spot it, you split your data into a training set and a validation set the model never trains on, and you watch both errors. At first both fall. When the validation error stops falling and rises again while the training error keeps falling, the growing gap is overfitting. The simplest fixes are more data, a smaller model, or early stopping, which means you stop where the validation error is lowest. The code (overfit.py, 13 lines of numpy, seed 44) fits polynomials of degree 1, 4 and 15 to 16 noisy points of sin(3x). Degree 1 misses the shape (validation error 0.233), degree 4 fits it well (0.056), and degree 15 runs through every point with a training error of 0.0 and a validation error of 29,914.926 on 100 new points. Honest notes: the training curves come from the same degree 15 model trained step by step with gradient descent for 1,000,000 epochs, where the validation error is lowest after epoch 798 and ends 112 times higher. #machinelearning #ai #overfitting #deeplearning #datascience

Comments

Comments from the original post on TikTok

No comments yet