AI engineering glossary · Deep learning core
What is NTK (neural tangent kernel)?
Neural tangent kernel (also called NTK): A theoretical tool showing that, for extremely wide networks with suitable scaling, training by gradient descent behaves like fitting a fixed kernel method. It helps analyse why overparameterised networks train and generalise.
Neural tangent kernel: an example
In the infinite-width limit, a network's predictions evolve as if governed by a fixed kernel matrix.
Related terms
More Deep learning core terms
- Neuron
- One-cycle policy
- Optimiser
- Orthogonal initialisation
- Output layer
- Parametric ReLU
- Recurrent neural network
- ReLU
Where you learn it
Neural tangent kernel is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.