AI engineering glossary · RAG vector search
What is synthetic test set?
Synthetic test set: An evaluation set whose questions and answers are generated by a language model from the documents themselves, saving manual labelling. It scales cheaply, but needs a human spot-check because generated questions can be too easy or unrealistic.
Synthetic test set: an example
A model writes 2 questions per chunk from 500 chunks, giving 1,000 test questions that are then sampled and reviewed.
Related terms
More RAG vector search terms
- Table extraction
- Text extraction
- Top-k retrieval
- Two-stage retrieval
- Vector database
- Vector index
- Vector normalisation
- Vector quantisation
Where you learn it
Synthetic test set is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.