AI engineering glossary · Inference serving optimisation
What is model cascade?
Model cascade: Trying a cheap model first and escalating to a stronger one only if the cheap model's answer seems unreliable, judged by a confidence score or a checker. It saves cost when most requests are easy.
Model cascade: an example
A small model answers first; if its confidence is below 0.7, the request is re-run on the large model.
Related terms
More Inference serving optimisation terms
Where you learn it
Model cascade is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.