Machine Learning · Chapter 6 of 40
Bias vs Variance
BIAS: error from wrong assumptions (too-simple model → underfit). VARIANCE: error from sensitivity to training data (too-complex model → overfit).
Good models balance both — the bias-variance tradeoff.
Example 1 (python)
# High bias -> underfit (linear model on curved data)
# High variance -> overfit (deep tree on tiny data)Diagnose by comparing train/test scores.
Example 2 (python)
# Reduce variance: more data, regularization, simpler model, ensemblesTechniques to lower variance.
Key points
- Bias: systematic error.
- Variance: sensitivity to noise.
- Total error = bias² + variance + noise.
- Aim to balance both.
💡 Note: Ensembles (random forests, boosting) reduce variance while keeping bias low.