AI engineering glossary · Generative media multimodal
What is realtime speech interaction?
Realtime speech interaction: Voice conversation with an AI system in which audio is streamed both ways with low delay, allowing interruption and natural turn-taking. It is implemented either by chaining recognition, language and synthesis or by a native speech model.
Realtime speech interaction: an example
A customer interrupts a voice agent mid-sentence and it stops speaking and responds to the new question.
Related terms
More Generative media multimodal terms
- Reparameterisation trick
- Residual vector quantisation
- Reverse diffusion process
- RNN-Transducer
- Sample rate
- Score distillation sampling
- Score-based generative model
- Short-time Fourier transform
Where you learn it
Realtime speech interaction is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.