AI engineering glossary · Data engineering
What is parquet?
Parquet (also called Apache Parquet): An open columnar file format for analytics that stores values column by column with compression, encodings and built-in statistics. It makes scans of a few columns from huge tables fast and small, and is read by Spark, pandas, DuckDB and most warehouses.
Parquet: an example
A 10 GB CSV of clickstream often shrinks to a fraction of its size as Parquet, and a query needing two columns reads only those.
Related terms
More Data engineering terms
- Partitioning
- Predicate pushdown
- Protocol Buffers
- Publish-subscribe
- Resilient Distributed Dataset
- Row-oriented storage
- Schema evolution
- Schema on read
Where you learn it
Parquet is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.