AI engineering glossary · LLM training alignment
What is base model?
Base model (also called Pretrained model, Raw model): A language model that has finished pretraining but has not been trained to follow instructions or chat. It simply continues text in the style it has seen, so a question may be answered with more questions. It is the starting point for instruction tuning, alignment and domain fine-tuning.
Base model: an example
Prompt a base model with 'What is GST?' and it may continue with 'What is VAT? What is income tax?' instead of answering.
Related terms
More LLM training alignment terms
- Bradley-Terry model
- Causal language modelling
- Chat template
- Chinchilla scaling
- Chosen and rejected responses
- Clipped surrogate objective
- Common Crawl
- Compute-optimal training
Where you learn it
Base model is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.