AI engineering glossary · NLP embeddings
What is PPMI (positive pointwise mutual information)?
Positive pointwise mutual information (also called PPMI, Positive PMI): A variant of pointwise mutual information that replaces all negative values with zero: PPMI = max(PMI, 0). Negative values are unreliable with limited data, so this version is preferred when building word vectors from co-occurrence counts.
Positive pointwise mutual information: an example
A PMI of -1.2 becomes 0, while a PMI of 2.3 stays 2.3.
Related terms
More NLP embeddings terms
- Relation extraction
- Semantic role labelling
- Semantic similarity
- Semantic textual similarity
- Sentence embedding
- Sentence segmentation
- Sentence-BERT
- SentencePiece
Where you learn it
Positive pointwise mutual information is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.