AI engineering glossary · Inference serving optimisation
What is model serving?
Model serving: Making a trained model available to applications, typically as a network API that receives inputs, runs the model and returns outputs reliably at the required speed and scale. It includes deployment, batching, scaling and monitoring.
Model serving: an example
A fraud model is wrapped in an HTTP service that answers within 50 ms for each payment request.
Related terms
More Inference serving optimisation terms
Where you learn it
Model serving is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.