AI engineering glossary · Transformers LLM architecture
What is top-k expert routing?
Top-k expert routing: A routing rule in which, for each token, only the k experts with the highest router scores are used and their outputs are combined with the normalised scores as weights. Common choices are k = 1 or 2 to keep compute low.
Top-k expert routing: an example
With k = 2, a token's output is 0.6 * expert_3(x) + 0.4 * expert_5(x).
Related terms
More Transformers LLM architecture terms
- Top-k sampling
- Top-p sampling
- Transformer
- Transformer block
- Transformer MLP block
- Transformer-XL
- Typical sampling
- Universal Transformer
Where you learn it
Top-k expert routing is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.