AI engineering glossary · Generative media multimodal
What is visual instruction tuning?
Visual instruction tuning: Fine-tuning a vision-language model on images paired with instructions and responses, so it follows user requests about pictures. The data are often generated with a strong language model to describe image content.
Visual instruction tuning: an example
Training examples show a chart image, the instruction 'Summarise the trend', and a written answer.
Related terms
More Generative media multimodal terms
- Visual question answering
- VITS
- Voice activity detection
- Voice cloning
- VQ-VAE
- Wasserstein GAN
- Watermarking generated content
- wav2vec 2.0
Where you learn it
Visual instruction tuning is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.