AI engineering glossary · LLM training alignment
What is rank (LoRA)?
Rank (LoRA) (also called LoRA rank): The inner dimension r of the two small matrices in a LoRA update. Higher rank gives the adapter more capacity and more parameters; low ranks such as 8 or 16 are common defaults. Alpha, a scaling constant, is set together with it.
Rank (LoRA): an example
Raising r from 8 to 64 multiplies the adapter's parameter count by 8 for the same layer.
Related terms
More LLM training alignment terms
- Reasoning model
- Reinforcement learning from AI feedback
- Reinforcement learning from human feedback
- Reinforcement learning with verifiable rewards
- Rejection sampling fine-tuning
- Reward hacking
- Reward model
- Safety fine-tuning
Where you learn it
Rank (LoRA) is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.