AI engineering glossary · LLM training alignment
What is PEFT (parameter-efficient fine-tuning)?
Parameter-efficient fine-tuning (also called PEFT): A group of methods that adapt a large pretrained model by training only a small number of added or selected parameters while the rest stay frozen. They cut memory, storage and compute, and let one base model serve many tasks with small swappable add-ons.
Parameter-efficient fine-tuning: an example
Instead of updating 7 billion weights, train about 20 million adapter weights for a legal-document task.
Related terms
More LLM training alignment terms
- PEFT library
- Pipeline parallelism
- Policy (reinforcement learning)
- Policy gradient
- Post-training
- Preference data
- Prefix tuning
- Pretraining
Where you learn it
Parameter-efficient fine-tuning is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.