AI engineering glossary · MLOps, LLMOps cloud
What is scale to zero?
Scale to zero: Automatically reducing a service to no running instances when it receives no traffic, and starting one again on the next request. It saves money for rarely used services, at the cost of a delay on the first request after idle time.
Scale to zero: an example
A demo model endpoint costs nothing overnight, but the first morning request waits 30 seconds while a container starts.
Related terms
More MLOps, LLMOps cloud terms
- Secrets management
- Security group
- Serverless
- Service level indicator
- Sidecar container
- Smoke test
- Staging environment
- Stateless service
Where you learn it
Scale to zero is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.