AI engineering glossary · Prompt context engineering
What is token counting?
Token counting: Measuring how many tokens a text will use, with the model's own tokenizer or a provider's counting tool, before sending it. Counts differ between models and languages, and Hindi or Tamil text often uses more tokens than English for the same meaning.
Token counting: an example
Run the tokenizer on a 3-page contract to confirm it fits in the 8,000-token input allowance.
Related terms
More Prompt context engineering terms
- Token pricing
- Tool choice
- Tool description
- Tool result message
- Tool schema
- Tree of Thoughts
- User prompt
- XML tags in prompts
Where you learn it
Token counting is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.