AI engineering glossary · Linear algebra
What is tensor rank?
Tensor rank (also called Tensor order): The number of axes of a tensor, also called its order. A scalar has rank 0, a vector rank 1, a matrix rank 2; this differs from matrix rank, which counts independent columns.
Tensor rank: an example
A tensor of shape (32, 128, 768), such as a batch of token embeddings, has rank 3.
Related terms
More Linear algebra terms
Where you learn it
Tensor rank is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.