AI engineering glossary · Evals guardrails AI security
What is content moderation?
Content moderation (also called moderation): Detecting and handling harmful or disallowed content, such as hate, harassment, sexual material, self-harm and violence, in what users send or models produce. It uses classifiers, policies and human review, and is often provided as a separate moderation service.
Content moderation: an example
A community app runs each post through a moderation model and holds those flagged as harassment for review.
Related terms
More Evals guardrails AI security terms
- Crescendo attack
- Cross-tenant leakage
- Data exfiltration
- Data masking
- Data poisoning
- DeepEval
- Defence in depth
- Denial of wallet
Where you learn it
Content moderation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.