2.7k citations · 2.9k across the 8 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.CL2021★ 14 cited
Efficient Large Scale Language Modeling with Mixtures of Experts
Mikel Artetxe, Shruti Bhosale, Naman Goyal +21
Mixture of Experts layers (MoEs) enable efficient scaling of language models through conditional computation. This paper presents a detailed empirical study of how autoregressive M…
cs.CL2021★ 77 cited
Few-shot Learning with Multilingual Language Models
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe +18
Large-scale generative language models such as GPT-3 are competitive few-shot learners. While these models are known to be able to jointly represent many different languages, their…