AI engineering glossary · Deep learning core
What is activation function?
Activation function (also called Non-linearity, Transfer function): A non-linear function applied to each neuron's weighted sum. Without it, stacked layers would collapse into one linear transformation; with it, a network can model curves and complex boundaries. Common choices include ReLU, sigmoid, tanh and GELU.
Activation function: an example
Applying ReLU to the values [-2, 0.5, 3] gives [0, 0.5, 3].
Related terms
More Deep learning core terms
- Adafactor
- Artificial neural network
- Autoencoder
- Autograd
- Backpropagation
- Backpropagation through time
- Backward pass
- Batch normalisation
Where you learn it
Activation function is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.