AI engineering glossary · MLOps, LLMOps cloud
What is stateless service?
Stateless service: A service that keeps no user-specific data in its own memory between requests, so any instance can answer any request. This makes it easy to add, remove and replace copies, which is why state is kept in databases or caches.
Stateless service: an example
Any of the ten API replicas can handle a user's next message because the conversation history lives in a shared database.
Related terms
- Load balancer
- Stateful application
- Autoscaling
- Twelve-factor app
More MLOps, LLMOps cloud terms
- Terraform
- Token usage tracking
- Virtual private cloud
- Workflow orchestration
- Alerting
- Amazon EC2
- Amazon ECR
- Amazon EKS
Where you learn it
Stateless service is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.