Lecture 20 – Decision Trees
by Josh Hug (Fall 2019)
Important: This lecture is taken from the Fall 2019 semester.
- The reference to Lecture 22 in 20.1 should be a reference to Lecture 19.
- The references to lec25-decision-trees.ipynb in 20.1 should be references to lec20-decision-trees.ipynb.
- The slides in 20.4 should say: “Bagging often isn’t enough to reduce model variance!” Without selecting a random subset of features at each split, the decision trees fitted on bagged data often look very similar to one another; this means that they make similar predictions. As a result, the ensemble of decision trees would still have low bias and high model variance. Selecting a random subset of features at each split helps reduce model variance by making the decision trees in the ensemble different from one another.
| Video | Quick Check | |
|---|---|---|
| 20.1 Decision tree basics. Decision trees in scikit-learn. |
20.1 | |
| 20.2 Overfitting and decision trees. |
20.2 | |
| 20.3 Decision tree generation. Finding the best split. Entropy and weighted entropy. |
20.3 | |
| 20.4 Restricting decision tree complexity. Preventing growth and pruning. Random forests and bagging. |
20.4 | |
| 20.5 Regression trees. Summary of decision trees, classification, and regression. |
20.5 |