AI engineering glossary · Deep learning core
What is focal loss?
Focal loss: A modified cross-entropy that shrinks the loss contribution of examples the model already gets right, multiplying it by (1 - p)^gamma. It focuses training on hard examples and helps when one class hugely outnumbers the others.
Focal loss: an example
In fraud detection with 1 fraud in 1,000 transactions, it keeps easy legitimate cases from dominating the gradient.
Related terms
- Cross-entropy loss
- Loss function
- Label smoothing
More Deep learning core terms
- Forward pass
- Fully connected layer
- Gated linear unit
- Gating mechanism
- GELU
- Gradient checkpointing
- Gradient norm
- Group normalisation
Where you learn it
Focal loss is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.