AI engineering glossary · LLM training alignment
What is LIMA?
LIMA: 'Less Is More for Alignment', a study that fine-tuned a large pretrained model on roughly one thousand carefully curated prompt-response pairs, without reinforcement learning, and found it performed competitively. It supports the view that data quality matters more than quantity for instruction tuning.
LIMA: an example
A 1,000-example curated set can rival one with 50,000 noisy examples for teaching an assistant style.
Related terms
More LLM training alignment terms
- Long-context fine-tuning
- LoRA
- Loss masking
- Loss spike
- Masked language modelling
- Maximal update parametrisation
- Megatron-LM
- Model collapse
Where you learn it
LIMA is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.