AI engineering glossary · MLOps, LLMOps cloud
What is token usage tracking?
Token usage tracking: Measuring how many input and output tokens each request, user or feature consumes, since language model providers charge per token. It supports budgeting, cost allocation, spotting runaway prompts and setting limits.
Token usage tracking: an example
A dashboard shows the summarisation feature uses 60% of monthly tokens, and one prompt is 4,000 tokens longer than needed.
Related terms
- LLM observability
- LLM cost management
- Rate limiting
- Cost allocation tags
More MLOps, LLMOps cloud terms
- Virtual private cloud
- Workflow orchestration
- Alerting
- Amazon EC2
- Amazon ECR
- Amazon EKS
- Amazon SageMaker
- Argo CD
Where you learn it
Token usage tracking is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.