AI engineering glossary · NLP embeddings
What is negative sampling?
Negative sampling: A training shortcut for Word2Vec that, for each true word-context pair, also draws a few random unrelated words as negative examples, and trains the model to tell real pairs from fake ones. It avoids computing a softmax over the entire vocabulary.
Negative sampling: an example
For each real pair (hot, chai), 5 random words such as "bridge" or "tax" are used as negatives.
Related terms
More NLP embeddings terms
- NLTK
- One-hot word vector
- Out-of-vocabulary word
- Padding token
- Part-of-speech tagging
- Pointwise mutual information
- Positive pointwise mutual information
- Relation extraction
Where you learn it
Negative sampling is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.