AI engineering glossary · Calculus optimisation
What is condition number?
Condition number (also called Ill-conditioning): The ratio of the largest to the smallest curvature of an objective, given at a minimum by the ratio of the extreme Hessian eigenvalues. A large value makes the surface a long narrow valley, forcing small steps and slow zig-zagging convergence.
Condition number: an example
For f(x, y) = x^2 + 100y^2, the Hessian eigenvalues are 2 and 200, so the condition number is 100 and plain gradient descent crawls.
Related terms
More Calculus optimisation terms
- Conjugate gradient method
- Constrained optimisation
- Constraint
- Continuity
- Convergence
- Convergence rate
- Convex function
- Convex optimisation
Where you learn it
Condition number is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.