Load What You Need: Smaller Versions of Multilingual BERT

Amine Abdaoui, Camille Pradel, Grégoire Sigel

Pre-trained Transformer-based models are achieving state-of-the-art results on a variety of Natural Language Processing data sets. However, the size of these models is often a draw…