Bilingual Alignment Pre-Training for Zero-Shot Cross-Lingual Transfer

Ziqing Yang, Wentao Ma, Yiming Cui +3

Multilingual pre-trained models have achieved remarkable performance on cross-lingual transfer learning. Some multilingual models such as mBERT, have been pre-trained on unlabeled…