AI engineering glossary · Transformers LLM architecture
What is GPT?
GPT (also called Generative Pre-trained Transformer): Generative Pre-trained Transformer: a family of decoder-only models trained to predict the next token on large text corpora. The name is now often used for the general approach of generative pretraining and for OpenAI's series of models built on it.
GPT: an example
A GPT-style model trained on web text can continue a prompt like 'Dear hiring manager,' into a full cover letter.
Related terms
More Transformers LLM architecture terms
- Greedy decoding
- Grouped-query attention
- Head dimension
- Hidden states
- Induction head
- Instruct model
- KV cache
- Large language model
Where you learn it
GPT is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.