AI engineering glossary · Evals guardrails AI security
What is synthetic test data?
Synthetic test data (also called synthetic evaluation data): Test cases generated by a language model or by scripts instead of collected from real users, used to expand coverage of edge cases, languages or rare failures cheaply. It should be reviewed by humans, since generated examples can be repetitive, unrealistic or wrong.
Synthetic test data: an example
To test a KYC assistant, a model generates 200 varied customer questions, including misspelt Hinglish, which a person then screens.
Related terms
More Evals guardrails AI security terms
- System prompt extraction
- Threat modelling
- Tool poisoning
- Tool-call accuracy
- Topical rails
- Toxicity classification
- Training data extraction
- TruthfulQA
Where you learn it
Synthetic test data is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.