AI engineering glossary · Transformers LLM architecture
What is decoder-only model?
Decoder-only model (also called Causal decoder): A Transformer with only a decoder stack and causal attention, trained to predict the next token. It generates text by repeatedly predicting and appending tokens. This is the design behind GPT-style and most current chat models.
Decoder-only model: an example
Given 'The capital of India is', a decoder-only model predicts ' New' then ' Delhi' one token at a time.
Related terms
More Transformers LLM architecture terms
Where you learn it
Decoder-only model is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.