AI engineering glossary · Deep learning core
What is sparse categorical cross-entropy?
Sparse categorical cross-entropy: The multi-class cross-entropy loss computed when labels are given as integer class indices rather than one-hot vectors. It gives the same value but avoids building large one-hot arrays, saving memory when there are many classes.
Sparse categorical cross-entropy: an example
With label 2 and predicted probabilities [0.1, 0.2, 0.7], the loss is -ln 0.7, about 0.36.
Related terms
More Deep learning core terms
- Spectral normalisation
- Stochastic depth
- Stochastic weight averaging
- Straight-through estimator
- SwiGLU
- Symmetry breaking
- Tanh
- Teacher forcing
Where you learn it
Sparse categorical cross-entropy is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.