LOLA -- An Open-Source Massively Multilingual Large Language Model

Nikit Srivastava, Denis Kuchelev, Tatiana Moteu Ngoli +5

This paper presents LOLA, a massively multilingual large language model trained on more than 160 languages using a sparse Mixture-of-Experts Transformer architecture. Our architect…