Vocabulary shapes cross-lingual variation of word-order learnability in language models

Jonas Mayer Martins, Jaap Jumelet, Viola Priesemann +1

Why do some languages like Czech permit free word order, while others like English do not? We address this question by pretraining transformer language models on a spectrum of synt…