One Form to Transfer Them All: Pretraining Multilingual Language Models Beyond Native Orthography

Muge Zhang, Aaron Jencks, Krishna Badikela +2

Multilingual language models transfer knowledge across languages through shared subword vocabulary, a mechanism that breaks down when related languages use different writing system…