AI engineering glossary · Data engineering
What is data pipeline?
Data pipeline: A chain of automated steps that moves data from one or more sources to a destination, applying checks and changes along the way. Pipelines replace manual copying, run on a schedule or on events, and are the basic building block of data engineering work.
Data pipeline: an example
A pipeline reads app logs every hour, filters out bots, aggregates page views per city and writes the counts to a dashboard table.
Related terms
More Data engineering terms
- Data profiling
- Data quality
- Data retention policy
- Data serialization
- Data skew
- Data SLA
- Data tiering
- Data validation
Where you learn it
Data pipeline is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.