AI engineering glossary · Inference serving optimisation
What is cost per token?
Cost per token (also called Cost per million tokens): The price of processing one token, either the fee charged by an API provider or your own hardware cost divided by tokens produced. It is normally quoted per million tokens, with input and output tokens priced differently.
Cost per token: an example
If a GPU costs Rs 200 per hour and generates 2 million tokens in that hour, the cost is about Rs 100 per million tokens.
Related terms
More Inference serving optimisation terms
Where you learn it
Cost per token is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.