AI engineering glossary · NLP embeddings
What is Kneser-Ney smoothing?
Kneser-Ney smoothing: A strong smoothing method for n-gram models that estimates a word's lower-order probability by how many different contexts it follows, not by its raw frequency. This stops words that are common only in fixed phrases from being over-predicted.
Kneser-Ney smoothing: an example
"Kong" is frequent but almost only follows "Hong", so Kneser-Ney gives it a low chance after other words.
Related terms
More NLP embeddings terms
- Massive Text Embedding Benchmark
- Matryoshka representation learning
- Mean pooling
- Morphology
- Multilingual embedding
- N-gram
- N-gram language model
- Named entity
Where you learn it
Kneser-Ney smoothing is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.