most citedDiversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer

3 citations · 9 across the 3 of their papers we have counts for

collaborators
Showing cs.CLShow all

2 papers · 1 filter

cs.CL2023★ 3 cited

Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer

Boan Liu, Liang Ding, Li Shen +4

The Mixture of Experts (MoE) has emerged as a highly successful technique in deep learning, based on the principle of divide-and-conquer to maximize model capacity without signific…

cs.CL2022★ 3 cited

Vega-MT: The JD Explore Academy Translation System for WMT22

Changtong Zan, Keqin Peng, Liang Ding +9

We describe the JD Explore Academy's submission of the WMT 2022 shared general translation task. We participated in all high-resource tracks and one medium-resource track, includin…