AI engineering glossary · Generative media multimodal
What is vision encoder?
Vision encoder: The image-processing component of a vision-language model that converts a picture into a sequence of feature vectors. It is commonly a vision transformer pretrained with a contrastive objective and often kept frozen.
Vision encoder: an example
A 336 x 336 image passes through the encoder and emerges as 576 patch feature vectors.
Related terms
More Generative media multimodal terms
- Vision-language model
- Visual grounding
- Visual instruction tuning
- Visual question answering
- VITS
- Voice activity detection
- Voice cloning
- VQ-VAE
Where you learn it
Vision encoder is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.