AI engineering glossary · Calculus optimisation
What is Gauss-Newton method?
Gauss-Newton method: An algorithm for non-linear least squares that approximates the Hessian by J^T J, using only first derivatives of the residuals. It avoids second derivatives and works well when residuals are small near the solution.
Gauss-Newton method: an example
Fitting the curve y = a * exp(b * x) to data points by repeatedly solving a linear least-squares problem built from the Jacobian of the residuals.
Related terms
More Calculus optimisation terms
- Genetic algorithm
- Global minimum
- Gradient
- Gradient accumulation
- Gradient checking
- Gradient clipping
- Gradient descent
- Hessian matrix
Where you learn it
Gauss-Newton method is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.