AI engineering glossary · Transformers LLM architecture
What is sequence length?
Sequence length (also called Sequence size): The number of tokens in a single input the model processes at once, including the prompt and, when generating, the output so far. Compute and memory for attention grow with it, and each model has a maximum it can handle.
Sequence length: an example
A 3-page document of about 1,500 words is roughly 2,000 tokens, so the sequence length is about 2,000.
Related terms
More Transformers LLM architecture terms
- Sequence-to-sequence model
- Shared expert
- Sinusoidal positional encoding
- Sliding window attention
- Sparse attention
- Sparse autoencoder
- State space model
- Switch Transformer
Where you learn it
Sequence length is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.