AI engineering glossary · Transformers LLM architecture
What is BART?
BART: An encoder-decoder Transformer pretrained by corrupting text, for example by deleting spans, shuffling sentences or masking tokens, and training the decoder to reconstruct the original. It suits generation tasks such as summarisation and translation.
BART: an example
A summarisation model fine-tuned from BART reads a 700-word news article and writes a 3-sentence summary.
Related terms
More Transformers LLM architecture terms
- Beam search
- BERT
- Bidirectional attention
- BigBird
- Capacity factor
- Causal masking
- Constrained decoding
- Context window
Where you learn it
BART is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.