Czert -- Czech BERT-like Model for Language Representation

Jakub Sido, Ondřej Pražák, Pavel Přibáň +3

This paper describes the training process of the first Czech monolingual language representation models based on BERT and ALBERT architectures. We pre-train our models on more than…