AI engineering glossary · Deep learning core
What is universal approximation theorem?
Universal approximation theorem: A mathematical result showing that a feedforward network with one hidden layer and enough neurons, using a suitable non-linear activation, can approximate any continuous function on a bounded region to any desired accuracy. It says such a network exists, not that training finds it.
Universal approximation theorem: an example
In principle a wide single-hidden-layer network can approximate sin(x) on [0, 10] to any accuracy.
Related terms
More Deep learning core terms
- Validation loss
- Vanishing gradient
- Weight decay
- Weight initialisation
- Weight normalisation
- Weight sharing
- Weight tying
- Xavier initialisation
Where you learn it
Universal approximation theorem is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.