AI engineering glossary · Data engineering
What is data skew?
Data skew: An uneven spread of data across partitions, where a few keys hold far more records than others, so a few workers do most of the work while the rest wait. It causes slow, straggling jobs and memory errors.
Data skew: an example
If 40 percent of orders have city = 'Unknown', the one worker handling that key becomes the bottleneck of the job.
Related terms
More Data engineering terms
Where you learn it
Data skew is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.