Exploiting Word Semantics to Enrich Character Representations of Chinese Pre-trained Models

Wenbiao Li, Rui Sun, Yunfang Wu

Most of the Chinese pre-trained models adopt characters as basic units for downstream tasks. However, these models ignore the information carried by words and thus lead to the loss…