192 citations · 233 across the 8 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.CL2021★ 1 cited
Ensembling of Distilled Models from Multi-task Teachers for Constrained Resource Language Pairs
Amr Hendy, Esraa A. Gad, Mohamed Abdelghaffar +4
This paper describes our submission to the constrained track of WMT21 shared news translation task. We focus on the three relatively low resource language pairs Bengali to and from…
cs.CL2021★ 33 cited
Scalable and Efficient MoE Training for Multitask Multilingual Models
Young Jin Kim, Ammar Ahmad Awan, Alexandre Muzio +6
The Mixture of Experts (MoE) models are an emerging class of sparsely activated deep learning models that have sublinear compute costs with respect to their parameters. In contrast…