Learning Mutually Informed Representations for Characters and Subwords

Yilin Wang, Xinyi Hu, Matthew R. Gormley

Most pretrained language models rely on subword tokenization, which processes text as a sequence of subword tokens. However, different granularities of text, such as characters, su…