AI engineering glossary · RAG vector search
What is scalar quantisation?
Scalar quantisation: Compressing vector values by storing each number with fewer bits, for example converting 32-bit floats to 8-bit integers. It cuts memory by about four times with small accuracy loss and is simpler than product quantisation.
Scalar quantisation: an example
Storing each of 1,536 dimensions as 1 byte instead of 4 reduces 1 million vectors from about 6 GB to 1.5 GB.
Related terms
More RAG vector search terms
- Score fusion
- Self-RAG
- Semantic cache
- Semantic chunking
- Sentence window retrieval
- Similarity threshold
- Small-to-big retrieval
- Sparse retrieval
Where you learn it
Scalar quantisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.