AI engineering glossary · Deep learning core
What is catastrophic forgetting?
Catastrophic forgetting (also called Catastrophic interference): The tendency of a network trained on a new task to abruptly lose performance on earlier tasks, because updated weights overwrite what was learned before. It is a central obstacle in continual learning and fine-tuning.
Catastrophic forgetting: an example
A model fine-tuned only on legal text may lose much of its general chat ability.
Related terms
More Deep learning core terms
- Cell state
- Checkpoint
- Cross-entropy loss
- Curriculum learning
- Cyclical learning rate
- Data parallelism
- Deep learning
- Deep neural network
Where you learn it
Catastrophic forgetting is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.