Training a Bilingual Language Model by Mapping Tokens onto a Shared Character Space

Aviad Rom, Kfir Bar

We train a bilingual Arabic-Hebrew language model using a transliterated version of Arabic texts in Hebrew, to ensure both languages are represented in the same script. Given the m…