How multilingual is Multilingual BERT?

Telmo Pires, Eva Schlinger, Dan Garrette

In this paper, we show that Multilingual BERT (M-BERT), released by Devlin et al. (2018) as a single language model pre-trained from monolingual corpora in 104 languages, is surpri…