AI engineering glossary · Inference serving optimisation
What is TCO (total cost of ownership)?
Total cost of ownership (also called TCO): The full cost of running a system over its life, not just the sticker price: hardware or cloud fees, power, staff time, monitoring, downtime and upgrades. It is the fair basis for comparing self-hosting with a paid API.
Total cost of ownership: an example
A cheap-looking self-hosted model becomes dearer than an API once two engineers' time and idle GPU hours are counted.
Related terms
More Inference serving optimisation terms
- Tree attention
- Triton Inference Server
- Unstructured pruning
- Verification step
- vLLM
- Warp
- Weight-only quantisation
- Zero-point
Where you learn it
Total cost of ownership is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.