AI engineering glossary · LLM training alignment
What is data mixture?
Data mixture (also called Data recipe): The proportions of different sources, such as web text, code, books and multilingual text, sampled during pretraining. Changing the mixture shifts what a model is good at, so it is tuned with small experiments.
Data mixture: an example
A mixture of 70 percent web, 15 percent code, 10 percent books and 5 percent Indian-language text.
Related terms
More LLM training alignment terms
- Debate (AI safety)
- DeepSpeed
- Direct preference optimisation
- Domain-adaptive pretraining
- DoRA
- Double quantisation
- Emergent abilities
- Evol-Instruct
Where you learn it
Data mixture is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.