AI engineering glossary · Prompt context engineering
What is needle in a haystack test?
Needle in a haystack test: An evaluation that hides one specific fact (the needle) at varying depths inside a long block of unrelated text (the haystack) and checks whether the model can retrieve it. It probes how well a model uses long inputs.
Needle in a haystack test: an example
Insert 'The secret code is 4172' at 60% depth in a 100,000-token text and ask the model for the code.
Related terms
More Prompt context engineering terms
- One-shot prompting
- Outlines library
- Output format specification
- Output parser
- Output repair
- Parallel tool calls
- Plan-and-Solve prompting
- Position bias
Where you learn it
Needle in a haystack test is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.