AI engineering glossary · Linear algebra
What is low-rank approximation?
Low-rank approximation: Replacing a matrix by another of much smaller rank that stays as close as possible to the original, typically built from the top singular values. It compresses data and reduces parameters, and underlies fine-tuning methods that store updates as products of thin matrices.
Low-rank approximation: an example
Approximating a 1000 x 1000 matrix by two factors of 1000 x 10 and 10 x 1000 stores 20,000 numbers instead of 1,000,000.
Related terms
More Linear algebra terms
- Lp norm
- LU decomposition
- Matrix
- Matrix multiplication
- Matrix norm
- Matrix-vector product
- Norm
- Normal equations
Where you learn it
Low-rank approximation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.