AI engineering glossary · LLM training alignment
What is full fine-tuning?
Full fine-tuning: Updating all of a pretrained model's weights on new data. It can give the best task fit but needs memory for weights, gradients and optimiser state, and produces a full-size model copy for every task.
Full fine-tuning: an example
Fully fine-tuning a 7B model in 16-bit needs multiple GPUs' worth of memory, while a LoRA run may fit on one.
Related terms
More LLM training alignment terms
- Fully sharded data parallel
- Generalised advantage estimation
- Goodhart's law
- Group relative policy optimisation
- Helpful, honest and harmless
- Hugging Face TRL
- Identity preference optimisation
- InstructGPT
Where you learn it
Full fine-tuning is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.