AI engineering glossary · Transformers LLM architecture
What is capacity factor?
Capacity factor: In a mixture-of-experts layer, the setting that fixes how many tokens each expert can accept per batch, as a multiple of the average even share. Tokens beyond an expert's capacity are dropped or passed through unchanged, so it trades quality against efficiency.
Capacity factor: an example
With 1,000 tokens and 8 experts the even share is 125; a capacity factor of 1.25 lets each expert take up to 156 tokens.
Related terms
More Transformers LLM architecture terms
- Causal masking
- Constrained decoding
- Context window
- Contrastive decoding
- Cross-attention
- Decoder
- Decoder-only model
- Decoding
Where you learn it
Capacity factor is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.