AI engineering glossary · Evals guardrails AI security
What is garak?
Garak (also called NVIDIA garak): An open-source vulnerability scanner for language models that runs many probes, such as prompt injection, jailbreaks, data leakage and toxicity, against a target and reports which weaknesses it found. It plays a role similar to network security scanners.
Garak: an example
Before deploying an internal assistant, a team runs the scanner against it and reviews a report listing the probe categories that succeeded.
Related terms
More Evals guardrails AI security terms
- Greedy Coordinate Gradient
- Groundedness
- Guardrail
- Guardrails AI
- HarmBench
- HELM
- HumanEval
- Improper output handling
Where you learn it
Garak is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.