AI engineering glossary · NLP embeddings
What is matryoshka representation learning?
Matryoshka representation learning (also called Matryoshka embeddings): A training method that makes the first k dimensions of an embedding useful on their own, for several values of k. Vectors can then be cut to a shorter length to save storage and speed search, with only a modest loss in quality.
Matryoshka representation learning: an example
A 1024-number embedding can be cut to its first 256 numbers for a cheaper index.
Related terms
More NLP embeddings terms
- Mean pooling
- Morphology
- Multilingual embedding
- N-gram
- N-gram language model
- Named entity
- Named entity recognition
- Natural language processing
Where you learn it
Matryoshka representation learning is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.