AI engineering glossary · Generative media multimodal
What is neural audio codec?
Neural audio codec: A learned compressor that encodes audio into a few discrete tokens per frame and decodes them back to sound. Its tokens let language-model-style networks generate speech and music. This compact discrete representation is what lets language-model-style networks generate speech and music efficiently.
Neural audio codec: an example
A codec turns 1 second of audio into 75 frames of 8 tokens each, and a transformer can predict those tokens.
Related terms
More Generative media multimodal terms
- Neural radiance field
- Neural vocoder
- Noise schedule
- Normalizing flow
- Object hallucination
- Omni model
- Outpainting
- Perceiver resampler
Where you learn it
Neural audio codec is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.