Sub-Character Tokenization for Chinese Pretrained Language Models

Chenglei Si, Zhengyan Zhang, Yingfa Chen +6

Tokenization is fundamental to pretrained language models (PLMs). Existing tokenization methods for Chinese PLMs typically treat each character as an indivisible token. However, th…