AI engineering glossary · NLP embeddings
What is subword regularisation?
Subword regularisation: A training trick that randomly samples different valid subword splits of the same text instead of always using the single best split. Showing the model several segmentations makes it more robust to spelling variation and noisy input.
Subword regularisation: an example
The word "unbelievable" is sometimes fed as ["un", "believable"] and sometimes as ["unbeliev", "able"] across epochs.
Related terms
More NLP embeddings terms
- Subword tokenisation
- Term frequency
- Text classification
- Text embedding model
- Text normalisation
- TF-IDF
- Token
- Token ID
Where you learn it
Subword regularisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.