Extending Multilingual BERT to Low-Resource Languages

Zihan Wang, Karthikeyan K, Stephen Mayhew +1

Multilingual BERT (M-BERT) has been a huge success in both supervised and zero-shot cross-lingual transfer learning. However, this success has focused only on the top 104 languages…