AI engineering glossary · Computer vision
What is CLS token?
CLS token (also called Class token): An extra learnable token prepended to the patch sequence in a vision transformer, whose final output is used as a summary of the whole image for classification. It has no image content of its own; attention lets it gather information from every patch.
CLS token: an example
After the last layer, the CLS vector goes through a linear layer to give 1,000 ImageNet class scores.
Related terms
More Computer vision terms
- Colour jitter
- Confidence score
- Connectionist temporal classification
- Contour detection
- Contrastive language-image pretraining
- Contrastive learning
- Convolution
- Convolutional neural network
Where you learn it
CLS token is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.