6 citations · 11 across the 3 of their papers we have counts for
3 papers
cs.LG2022
StableMoE: Stable Routing Strategy for Mixture of Experts
Damai Dai, Li Dong, Shuming Ma +4
The Mixture-of-Experts (MoE) technique can scale up the model size of Transformers with an affordable computational overhead. We point out that existing learning-to-route MoE metho…
cs.CL2021★ 5 cited
Consistency Regularization for Cross-Lingual Fine-Tuning
Bo Zheng, Li Dong, Shaohan Huang +7
Fine-tuning pre-trained cross-lingual language models can transfer task-specific supervision from one language to the others. In this work, we propose to improve cross-lingual fine…
cs.CL2020★ 6 cited
Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension
Bo Zheng, Haoyang Wen, Yaobo Liang +5
Natural Questions is a new challenging machine reading comprehension benchmark with two-grained answers, which are a long answer (typically a paragraph) and a short answer (one or…