Data Science ยท Chapter 13 of 43
Handling Missing Values
Missing values are represented by NaN. Detect with `isna`, drop with `dropna`, fill with `fillna` or scikit-learn's SimpleImputer.
Always handle missing values on the TRAIN set first, then apply the same transform to test.
Example 1 (python)
import pandas as pd
df = pd.read_csv('data.csv')
print(df.isna().sum())Missing count per column.
Example 2 (python)
df['age'] = df['age'].fillna(df['age'].median())
df = df.dropna(subset=['target'])Fill features, drop rows missing the target.
Key points
- Detect with isna().
- Fill numeric with median, categorical with mode.
- Drop rows only when necessary.
- Never leak test values into imputation.
๐ก Note: Adding a boolean `was_missing` column can add real signal โ sometimes the missingness itself is predictive.