Machine Learning · Chapter 12 of 40
K-Nearest Neighbors (KNN)
KNN classifies a point by taking a majority vote of its K nearest neighbours in the training set.
No real training — the model just stores the data (lazy learner).
Example 1 (python)
from sklearn.neighbors import KNeighborsClassifier
m = KNeighborsClassifier(n_neighbors=5).fit(X_train, y_train)
print(m.predict(X_test[:3]))Output
[0, 1, 1]Vote of 5 nearest neighbours.
Example 2 (python)
# Scale features before KNN — distance is sensitive to scaleAlways standardise before KNN.
Key points
- Lazy learner — no real training.
- Requires a distance metric.
- Sensitive to feature scale.
- K controls smoothness.
💡 Note: Small K → sensitive to noise. Large K → oversmoothed. Try odd values in cross-validation.