A Comparison of Architectures and Pretraining Methods for Contextualized Multilingual Word Embeddings

Niels van der Heijden, Samira Abnar, Ekaterina Shutova

The lack of annotated data in many languages is a well-known challenge within the field of multilingual natural language processing (NLP). Therefore, many recent studies focus on z…