AI engineering glossary · Transformers LLM architecture
What is SSM (state space model)?
State space model (also called SSM): A sequence model that keeps a fixed-size hidden state updated at each step by learned linear dynamics, and can be computed either recurrently or as a convolution. It handles long sequences with linear cost, making it an alternative to attention.
State space model: an example
An S4-style layer processes a 100,000-step signal with a state of a few hundred numbers instead of comparing every pair of steps.
Related terms
More Transformers LLM architecture terms
- Switch Transformer
- T5
- Temperature
- Test-time compute
- Top-k expert routing
- Top-k sampling
- Top-p sampling
- Transformer
Where you learn it
State space model is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.