AI engineering glossary · Data engineering
What is backpressure?
Backpressure: A mechanism by which a slow consumer signals upstream stages to slow down, or a system limits intake, so buffers do not overflow when data arrives faster than it can be processed.
Backpressure: an example
When a database sink slows down, the stream job reads fewer messages from Kafka instead of running out of memory.
Related terms
More Data engineering terms
- Batch processing
- BigQuery
- Bucketing
- Change data capture
- Checkpointing
- Columnar storage
- Compaction
- Consumer group
Where you learn it
Backpressure is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.