AI engineering glossary · LLM training alignment
What is foundation model?
Foundation model: A large model trained on broad data at scale so that it can be adapted to many downstream tasks, rather than built for one task. Language models, image models and multimodal models can all be foundation models. Their value is that one expensive training run supports many products through prompting or fine-tuning.
Foundation model: an example
A single pretrained model can be prompted to summarise Hindi news, or fine-tuned to answer questions on bank policy documents.
Related terms
More LLM training alignment terms
- FP8 training
- Full fine-tuning
- Fully sharded data parallel
- Generalised advantage estimation
- Goodhart's law
- Group relative policy optimisation
- Helpful, honest and harmless
- Hugging Face TRL
Where you learn it
Foundation model is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.