Lecture 16 – Bias and Variance
Presented by Isaac Schmidt
Content by Isaac Schmidt, Andrew Bray, Suraj Rampure, Ani Adhikari
Important: You may want to review Lecture 3 for a refresher on random variables.
The data generation process, and the assumptions we make when choosing a model.
Continuous random variables. Expectation of a function of a random variable. Bias, revisited.
Variance of a random variable. Linear transformations of variance. Covariance and correlation.
Bias and variance in modeling. Simulation of different models with different samples. Components of prediction error.
Risk. Decomposition of model risk into the components of prediction error.
The Bias-Variance decomposition. Reasons and remedies for the different components of error.
Overfitting and underfitting. Complexity, and the bias-variance tradeoff.