AI engineering glossary · Transformers LLM architecture
What is autoregressive model?
Autoregressive model (also called AR model): A model that generates or scores a sequence by factorising its probability into a product of each element given the earlier ones. In language modelling it means predicting tokens left to right, using the model's own outputs as inputs when generating.
Autoregressive model: an example
P('I love India') = P(I) * P(love | I) * P(India | I love).
Related terms
More Transformers LLM architecture terms
Where you learn it
Autoregressive model is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.