AI engineering glossary · Data engineering
What is schema evolution?
Schema evolution: The ability of a data format or table to cope with changes to its structure over time, such as added, renamed or retyped columns, while old and new data remain readable. Rules decide which changes are compatible.
Schema evolution: an example
A new nullable column loyalty_tier is added to an Avro schema, and old files still read fine with the value treated as null.
Related terms
More Data engineering terms
- Schema on read
- Schema on write
- Sensor
- Shuffle
- Slowly changing dimension
- Small files problem
- Snowflake
- Snowflake schema
Where you learn it
Schema evolution is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.