Machine Learning · Chapter 20 of 40
Feature Scaling
Many models (KNN, SVM, neural nets, PCA) are sensitive to feature scale. STANDARDIZATION (mean 0, std 1) or MIN-MAX SCALING (0..1) fixes it.
Tree-based models don't need scaling.
Example 1 (python)
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train_s = sc.fit_transform(X_train)
X_test_s = sc.transform(X_test)Fit on TRAIN only, transform both.
Example 2 (python)
from sklearn.preprocessing import MinMaxScaler
X_scaled = MinMaxScaler().fit_transform(X)Min-max squishes to [0, 1].
Key points
- Standardization: mean 0, std 1.
- Min-max: values in [0, 1].
- Trees don't need scaling.
- Fit on training data only.
💡 Note: Never fit the scaler on the test set — that's data leakage.