AI engineering glossary · Generative media multimodal
What is early fusion?
Early fusion: A multimodal design that combines modalities at the input level, with images, audio and text converted to tokens and processed together by one model from the start. It allows deep cross-modal interaction.
Early fusion: an example
Image patches and text tokens are placed in a single sequence and passed through the same transformer layers.
Related terms
More Generative media multimodal terms
- Evidence lower bound
- FastSpeech
- Flamingo
- Flow matching
- Forward diffusion process
- Fréchet Inception Distance
- Generative adversarial network
- Grapheme-to-phoneme
Where you learn it
Early fusion is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.