AI engineering glossary · Linear algebra
What is normal equations?
Normal equations: The equations X^T X b = X^T y whose solution gives the least-squares coefficients. They express that the residual must be perpendicular to every column of X, but forming X^T X can worsen numerical problems.
Normal equations: an example
For X with columns [1, 1, 1] and [1, 2, 3] and y = [2, 3, 5], solving the equations gives intercept 0.33 and slope 1.5.
Related terms
More Linear algebra terms
- Null space
- Orthogonal matrix
- Orthogonal projection
- Orthogonal vectors
- Orthonormal basis
- Outer product
- Permutation matrix
- Pivot
Where you learn it
Normal equations is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.