Pre-training Polish Transformer-based Language Models at Scale

Sławomir Dadas, Michał Perełkiewicz, Rafał Poświata

Transformer-based language models are now widely used in Natural Language Processing (NLP). This statement is especially true for English language, in which many pre-trained models…