WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models

Benjamin Minixhofer, Fabian Paischer, Navid Rekabsaz

Large pretrained language models (LMs) have become the central building block of many NLP applications. Training these models requires ever more computational resources and most of…