most citedA Loss Curvature Perspective on Training Instability in Deep Learning

6 citations · 6 across the 2 of their papers we have counts for

collaborators
Showing cs.CLShow all

1 paper · 1 filter

cs.CL2021

Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference

Sneha Kudugunta, Yanping Huang, Ankur Bapna +4

Sparse Mixture-of-Experts (MoE) has been a successful approach for scaling multilingual translation models to billions of parameters without a proportional increase in training com…