AI engineering glossary · RAG vector search
What is chunk size?
Chunk size: The target length of each chunk, measured in tokens, characters or sentences. Small chunks give precise matches but lose surrounding context; large chunks keep context but dilute the embedding and use more of the model's context window. It is one of the most tuned RAG settings.
Chunk size: an example
Moving from 1,000-token chunks to 300-token chunks raised the share of questions whose answer appeared in the top 5 results from 68% to 74% on one test set.
Related terms
More RAG vector search terms
- Citation
- ColBERT
- Context ordering
- Context precision
- Context recall
- Context relevance
- Contextual chunk enrichment
- Contextual retrieval
Where you learn it
Chunk size is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.