Problem 13 - Olympiad

When using batch normalization, during which phase are running means and variances updated?

A. Forward pass of each mini-batchB. Backward pass of each mini-batchC. Only during validationD. Only after training ends

Correct: A

Batch norm computes mini-batch statistics for normalization during training and updates exponential-moving-average mean/variance to use at inference.