AI engineering glossary · NLP embeddings
What is PMI (pointwise mutual information)?
Pointwise mutual information (also called PMI): A score of how much more often two words occur together than they would if independent: PMI(x, y) = log( P(x, y) / (P(x) P(y)) ). Positive values mean association and negative mean avoidance. It is used for collocations and for weighting co-occurrence counts.
Pointwise mutual information: an example
If P(x, y) = 0.002 while P(x) = 0.01 and P(y) = 0.02, PMI = log(0.002 / 0.0002) = log 10, about 2.3 in natural log.
Related terms
More NLP embeddings terms
- Positive pointwise mutual information
- Relation extraction
- Semantic role labelling
- Semantic similarity
- Semantic textual similarity
- Sentence embedding
- Sentence segmentation
- Sentence-BERT
Where you learn it
Pointwise mutual information is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.