EstBERT: A Pretrained Language-Specific BERT for Estonian

Hasan Tanvir, Claudia Kittask, Sandra Eiche +1

This paper presents EstBERT, a large pretrained transformer-based language-specific BERT model for Estonian. Recent work has evaluated multilingual BERT models on Estonian tasks an…