AI engineering glossary · Transformers LLM architecture
What is encoder-decoder model?
Encoder-decoder model (also called Encoder-decoder architecture): A Transformer with two stacks: an encoder that reads the full input and a decoder that generates the output while attending to it. Suited to tasks that map one sequence to another, such as translation and summarisation. T5 and BART are well-known examples.
Encoder-decoder model: an example
A summariser encodes a 1,000-word article and its decoder writes a 60-word summary.
Related terms
More Transformers LLM architecture terms
- Encoder-only model
- Expert
- Expert parallelism
- Fill-in-the-middle
- FlashAttention
- Frequency and presence penalty
- GPT
- Greedy decoding
Where you learn it
Encoder-decoder model is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.