AI engineering glossary · Generative media multimodal
What is RVQ (residual vector quantisation)?
Residual vector quantisation (also called RVQ): A compression method that applies several codebooks in sequence, each one encoding the error left by the previous one. More codebooks give higher fidelity, and the stack of indices per frame forms the tokens used by neural audio codecs.
Residual vector quantisation: an example
A codec uses 8 codebooks per frame: the first captures the coarse sound and later ones add fine detail, so dropping the last few lowers bitrate.
Related terms
More Generative media multimodal terms
- Reverse diffusion process
- RNN-Transducer
- Sample rate
- Score distillation sampling
- Score-based generative model
- Short-time Fourier transform
- SigLIP
- Speaker diarisation
Where you learn it
Residual vector quantisation is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.