Problem 12 - Olympiad

Which statement about the Universal Approximation Theorem is true?

A. It guarantees zero training error for any neural networkB. It applies only to ReLU networksC. A feed-forward network with one sufficiently large hidden layer can approximate any continuous function on compact setsD. It is irrelevant for convolutional networks

Correct: C

The theorem states that given enough hidden units (and non-linear activation), even a shallow network can represent any continuous function to arbitrary accuracy, although it says nothing about learnability or generalization.