AI engineering glossary · MLOps, LLMOps cloud
What is resource requests and limits?
Resource requests and limits: Settings on a Kubernetes container stating how much CPU, memory or GPU it is guaranteed (request) and the most it may use (limit). Requests guide scheduling onto nodes; exceeding a memory limit gets the container killed, so values need careful tuning.
Resource requests and limits: an example
A model-server container requests 4 CPUs and 16 GiB memory and is limited to 8 GiB more; going past the memory limit triggers an out-of-memory kill.
Related terms
- Kubernetes pod
- Resource quota
- Kubernetes node
- Horizontal Pod Autoscaler
More MLOps, LLMOps cloud terms
- Role-based access control
- Rolling update
- Scale to zero
- Secrets management
- Security group
- Serverless
- Service level indicator
- Sidecar container
Where you learn it
Resource requests and limits is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.