AI engineering glossary · NLP embeddings
What is zipf's law?
Zipf's law: An empirical pattern in language: a word's frequency is roughly inversely proportional to its rank in the frequency table, freq ~ 1 / rank. A few words are extremely common while most words are rare, which explains long-tailed vocabularies and sparse counts.
Zipf's law: an example
If the most common word appears 60,000 times, the second most common appears about 30,000 times and the tenth about 6,000.
Related terms
More NLP embeddings terms
Where you learn it
Zipf's law is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.