AI engineering glossary · Deep learning core
What is leCun initialisation?
LeCun initialisation: A scheme that draws weights with variance 1 / fan_in so that layer outputs keep roughly unit variance. It is the recommended starting point for SELU networks and an older basis for later methods.
LeCun initialisation: an example
A layer with 100 inputs uses weights with standard deviation 0.1.
Related terms
More Deep learning core terms
- Linear scaling rule
- Lion optimiser
- Log-sum-exp trick
- Logits
- Loss scaling
- Lottery ticket hypothesis
- LSTM
- Maxout
Where you learn it
LeCun initialisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.