AI engineering glossary · Data engineering
What is hadoop?
Hadoop: An open-source framework for storing and processing very large datasets across clusters of ordinary machines, made up of a distributed file system, a resource manager and the MapReduce engine. It defined early big data practice.
Hadoop: an example
An older enterprise keeps years of logs in a Hadoop cluster and queries them with Hive.
Related terms
More Data engineering terms
- HDFS
- Hive
- Idempotency
- Incremental load
- JSON Lines
- Kappa architecture
- Lambda architecture
- Late-arriving data
Where you learn it
Hadoop is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.