AI engineering glossary · Data engineering
What is incremental load?
Incremental load (also called Delta load): Loading only the data that is new or changed since the last run, tracked using timestamps, IDs or change logs, rather than copying everything. It is faster and cheaper than repeated full loads.
Incremental load: an example
Each night the job copies rows with updated_at later than the previous run's high-water mark.
Related terms
More Data engineering terms
- JSON Lines
- Kappa architecture
- Lambda architecture
- Late-arriving data
- Lazy evaluation
- MapReduce
- Master data management
- Medallion architecture
Where you learn it
Incremental load is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.