The birth of Romanian BERT

Stefan Daniel Dumitrescu, Andrei-Marius Avram, Sampo Pyysalo

Large-scale pretrained language models have become ubiquitous in Natural Language Processing. However, most of these models are available either in high-resource languages, in part…