AI engineering glossary · Evaluation metrics
What is BLEU?
BLEU (also called Bilingual Evaluation Understudy, BLEU score): A metric for machine translation that compares a system's output to one or more human reference translations. It computes modified n-gram precision (typically for 1- to 4-word sequences), combines them with a geometric mean and multiplies by a brevity penalty for outputs that are too short. Scores run 0 to 1, or 0 to 100.
BLEU: an example
A Hindi-to-English system output shares many 1- and 2-word sequences with the reference but few 4-word ones, so its BLEU is moderate. Very short outputs get the brevity penalty.
Related terms
More Evaluation metrics terms
- Brier score
- Calibration
- Calinski-Harabasz index
- Catalogue coverage
- Character error rate
- chrF
- CIDEr
- Cohen's kappa
Where you learn it
BLEU is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.