AI engineering glossary · NLP embeddings
What is wordPiece?
WordPiece: A subword tokenisation algorithm that builds its vocabulary by merging pieces chosen to maximise the likelihood of the training data. Continuation pieces are marked with a prefix such as ##. It is the tokeniser used by BERT-style models.
WordPiece: an example
"playing" may be split into ["play", "##ing"], where ## shows the piece continues the previous word.
Related terms
More NLP embeddings terms
- Zipf's law
- Agglutinative language
- Backoff
- Bi-encoder
- Bidirectional LSTM-CRF
- Bigram
- BILOU tagging
- BIO tagging
Where you learn it
WordPiece is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.