When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer

Ameet Deshpande, Partha Talukdar, Karthik Narasimhan

While recent work on multilingual language models has demonstrated their capacity for cross-lingual zero-shot transfer on downstream tasks, there is a lack of consensus in the comm…