AI engineering glossary · Deep learning core
What is xavier initialisation?
Xavier initialisation (also called Glorot initialisation): A scheme that sets the spread of initial weights from the number of inputs and outputs of a layer, roughly variance = 2 / (fan_in + fan_out), so that signal size stays similar forwards and backwards. It suits tanh and sigmoid activations.
Xavier initialisation: an example
A layer with 256 inputs and 256 outputs draws weights with variance 2/512, about 0.0039.
Related terms
More Deep learning core terms
- Activation function
- Adafactor
- Artificial neural network
- Autoencoder
- Autograd
- Backpropagation
- Backpropagation through time
- Backward pass
Where you learn it
Xavier initialisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.