AI engineering glossary · Prompt context engineering
What is prompt regression testing?
Prompt regression testing: Running a fixed set of test inputs with expected outcomes every time a prompt or model changes, to catch cases that used to work and now fail. It is the prompt equivalent of unit tests and is often run in continuous integration.
Prompt regression testing: an example
After editing a prompt, 150 saved test tickets are rerun and the build fails if accuracy drops below 90%.
Related terms
More Prompt context engineering terms
- Prompt sensitivity
- Prompt template
- Prompt versioning
- Pydantic validation of LLM output
- Quote-first prompting
- ReAct prompting
- Reasoning effort
- Recency bias in prompts
Where you learn it
Prompt regression testing is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.