Skip navigation
➡️ Machine Learning from Scratch part 18: Decision trees Our bank is simulated.
0:00 / 1:53

➡️ Machine Learning from Scratch part 18: Decision trees Our bank is simulated.

19,100 views #machinelearning #ai #decisiontree

➡️ Machine Learning from Scratch part 18: Decision trees Our bank is simulated. A small program of ours made up 500 customers, each with a monthly income, a number of missed payments and whether they paid their loan back. No real people and no real loans. How it works: A decision tree makes a decision by asking a series of simple yes or no questions. To pick a question, our code tries every possible cut on income and on missed payments and keeps the one that leaves the least mixed groups, measured with the Gini impurity (zero when everyone in a group has the same answer). Then the same function runs again on each half until the tree is two questions deep, and each leaf answers with what most of its customers did. Trained on 400 customers, it found almost the example from the start of the video on its own: is the income above 3,020, and if so, did the customer miss more than one payment? On 100 customers it had never seen, it got 82 right. Without a depth limit, the tree grows 12 questions deep and gets 398 of its 400 old customers right, yet only 79 of the new ones. It memorized the surprises in its data, the overfitting from part 4. Why it's used so much: You can read every decision it makes. Random forests and gradient boosting libraries like XGBoost and LightGBM are built from many of these trees, and a NeurIPS 2022 benchmark on 45 datasets found that tree-based models still outperform deep learning on typical medium-sized tabular data. Every number in the video comes from the real run of the program shown. The forest picture grows the same kind of tree on random samples of our customers. #machinelearning #ai #decisiontree #python #numpy

Comments

Comments from the original post on TikTok

No comments yet