Machine Learning · Chapter 13 of 40
Decision Trees
A DECISION TREE splits data by asking yes/no questions on features. Easy to visualise and interpret.
Single trees overfit easily — usually paired with ensembles (random forest, gradient boosting).
Example 1 (python)
from sklearn.tree import DecisionTreeClassifier
m = DecisionTreeClassifier(max_depth=4).fit(X_train, y_train)
print(m.score(X_test, y_test))Output
0.83Limit depth to reduce overfitting.
Example 2 (python)
from sklearn.tree import export_text
print(export_text(m, feature_names=list(X.columns))[:200])Output
|--- feature_0 <= 5.0 ...Trees are inspectable.
Key points
- Splits data with yes/no questions.
- Very interpretable.
- Prone to overfit — control depth.
- Base for random forests and XGBoost.
💡 Note: Set `max_depth` or `min_samples_leaf` to prevent trees from memorising the training data.