AI engineering glossary · Deep learning core
What is orthogonal initialisation?
Orthogonal initialisation: A scheme that starts a weight matrix as an orthogonal matrix, one whose columns are perpendicular unit vectors. Such matrices preserve vector length, so signals neither grow nor shrink through many layers or recurrent steps.
Orthogonal initialisation: an example
Initialising a 256x256 recurrent weight matrix orthogonally helps keep its repeated products stable.
Related terms
More Deep learning core terms
Where you learn it
Orthogonal initialisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.