Soft Language Clustering for Multilingual Model Pre-training

Jiali Zeng, Yufan Jiang, Yongjing Yin +5

Multilingual pre-trained language models have demonstrated impressive (zero-shot) cross-lingual transfer abilities, however, their performance is hindered when the target language…