DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionaries

Aditi Chaudhary, Karthik Raman, Krishna Srinivasan +1

Pre-trained multilingual language models such as mBERT have shown immense gains for several natural language processing (NLP) tasks, especially in the zero-shot cross-lingual setti…