Machine Learning · Chapter 33 of 40

Loss Functions

LOSS measures how wrong the model is. Regression: MSE, MAE. Binary classification: BINARY CROSS-ENTROPY. Multi-class: CATEGORICAL CROSS-ENTROPY.

Training aims to minimise the loss.

Example 1 (python)

# Regression
loss = 'mse'

# Binary
loss = 'binary_crossentropy'

# Multi-class
loss = 'sparse_categorical_crossentropy'

Pick to match the task.

Example 2 (python)

# Custom loss example
import tensorflow as tf
def mse(y_true, y_pred):
    return tf.reduce_mean(tf.square(y_true - y_pred))

You can write custom losses.

Key points

💡 Note: Match the loss to the last-layer activation: sigmoid + binary cross-entropy, softmax + categorical cross-entropy.