AI engineering glossary · Prompt context engineering
What is max tokens?
Max tokens (also called max_tokens): A request parameter that limits the number of tokens the model may generate in its reply. It controls cost and delay, but a low value can cut an answer off mid-sentence. It does not limit the size of the input.
Max tokens: an example
max_tokens=300 for a short summary; if the model needs more it stops at 300 with a 'length' finish reason.
Related terms
More Prompt context engineering terms
- Message roles
- Meta-prompting
- Multi-turn conversation
- Needle in a haystack test
- One-shot prompting
- Outlines library
- Output format specification
- Output parser
Where you learn it
Max tokens is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.