AI engineering glossary · NLP embeddings
What is smoothing?
Smoothing (also called Language model smoothing): A family of techniques that move a little probability from seen word sequences to unseen ones, so that no valid sequence gets probability zero. Without it, a single unseen n-gram would make an entire sentence impossible under the model.
Smoothing: an example
After smoothing, a bigram never seen in training gets a small probability such as 0.0001 instead of exactly 0.
Related terms
More NLP embeddings terms
- spaCy
- Special tokens
- Spelling correction
- Static embedding
- Subword regularisation
- Subword tokenisation
- Term frequency
- Text classification
Where you learn it
Smoothing is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.