Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language

Yuri Kuratov, Mikhail Arkhipov

The paper introduces methods of adaptation of multilingual masked language models for a specific language. Pre-trained bidirectional language models show state-of-the-art performan…