Universal approximation theorem

Explanation

Any neural network architecture aims at finding a function $y=f(x)$ that can map some given inputs $x$ to outputs $y$. $f(x)$ can be arbitrarily complex.

UAT states that NNs have a universality property where no matter the function $f(x)$, there is a neural network that can approach the function with a given precision.

This means that the problems are not limited to a specific type of function or problem, being able to model a wide range of relationships between inputs and outputs.

Neural networks may not necessarily find the exact function, but are able to achieve an accurate approximation that gets arbitrarily close to the true function.

• ? Does this apply to non-continuous functions?

Caveats

• The model might be unfeasibly large
• The model is not guaranteed to generalise