Training Multilingual Pre-trained Language Model with Byte-level Subwords

Junqiu Wei, Qun Liu, Yinpeng Guo +1

The pre-trained language models have achieved great successes in various natural language understanding (NLU) tasks due to its capacity to capture the deep contextualized informati…