AI engineering glossary · Data engineering
What is backfill?
Backfill: Running a pipeline over past time periods to fill in or correct historical data, for instance after a bug fix, a new column or a late source. Good pipelines are parameterised by date so this is simple.
Backfill: an example
After fixing a tax calculation, the team re-runs the daily job for the last 90 days.
Related terms
More Data engineering terms
- Backpressure
- Batch processing
- BigQuery
- Bucketing
- Change data capture
- Checkpointing
- Columnar storage
- Compaction
Where you learn it
Backfill is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.