AI engineering glossary · NLP embeddings
What is padding token?
Padding token (also called Pad token): A special filler token appended to shorter sequences so every sequence in a batch has the same length. An attention mask or similar signal tells the model to ignore these positions so they do not influence results.
Padding token: an example
Batching sequences of lengths 5 and 8 pads the first with three <pad> tokens to reach length 8.
Related terms
More NLP embeddings terms
- Part-of-speech tagging
- Pointwise mutual information
- Positive pointwise mutual information
- Relation extraction
- Semantic role labelling
- Semantic similarity
- Semantic textual similarity
- Sentence embedding
Where you learn it
Padding token is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.