AI engineering glossary · Generative media multimodal
What is VQ-VAE?
VQ-VAE (also called Vector-quantised VAE): A VAE variant whose latent codes are discrete: the encoder output is snapped to the nearest entry in a learned codebook of vectors. This gives images or audio a token-like representation that autoregressive models can predict, and avoids posterior collapse.
VQ-VAE: an example
A 256 x 256 image becomes a 32 x 32 grid of indices into a codebook of 8192 vectors, which a transformer can then model as a sequence.
Related terms
More Generative media multimodal terms
- Wasserstein GAN
- Watermarking generated content
- wav2vec 2.0
- WaveNet
- Whisper
- Zero-shot TTS
- 3D Gaussian splatting
- Acoustic model
Where you learn it
VQ-VAE is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.