AI engineering glossary · Transformers LLM architecture
What is log probabilities?
Log probabilities (also called Logprobs): The natural logarithms of the probabilities the model assigns to tokens. Many APIs can return them for generated and alternative tokens, and they are used to gauge confidence, rank outputs, compute perplexity and build classifiers on model output.
Log probabilities: an example
A token with probability 0.5 has a log probability of about -0.69; one with 0.01 has about -4.6.
Related terms
More Transformers LLM architecture terms
- Logit lens
- Long-context model
- Longformer
- Mamba
- Maximum output tokens
- Mechanistic interpretability
- Min-p sampling
- Mixtral
Where you learn it
Log probabilities is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.