AI engineering glossary · NLP embeddings
What is infoNCE loss?
InfoNCE loss (also called NT-Xent, Contrastive cross-entropy loss): A contrastive loss that, for each anchor, treats the true positive as the correct class among many candidates and applies softmax cross-entropy over their similarity scores divided by a temperature. Lower loss means the positive scores higher than all negatives.
InfoNCE loss: an example
With one positive and 7 negatives per query, the loss is the cross-entropy of picking the positive among 8 candidates.
Related terms
More NLP embeddings terms
- Intent classification
- Inverse document frequency
- Kneser-Ney smoothing
- Massive Text Embedding Benchmark
- Matryoshka representation learning
- Mean pooling
- Morphology
- Multilingual embedding
Where you learn it
InfoNCE loss is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.