AI engineering glossary · MLOps, LLMOps cloud
What is LLM evaluation pipeline?
LLM evaluation pipeline: An automated process that runs a fixed set of test prompts and scoring methods against each new prompt, model or retrieval change, and reports quality before release. It is the LLM counterpart of regression testing.
LLM evaluation pipeline: an example
On each pull request, 200 saved questions are answered and scored, and the merge is blocked if accuracy falls below 85%.
Related terms
- LLMOps
- Regression test
- Continuous integration
- Prompt versioning
More MLOps, LLMOps cloud terms
Where you learn it
LLM evaluation pipeline is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.