8 citations · 23 across the 13 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.CL2023★ 3 cited
Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer
Boan Liu, Liang Ding, Li Shen +4
The Mixture of Experts (MoE) has emerged as a highly successful technique in deep learning, based on the principle of divide-and-conquer to maximize model capacity without signific…
cs.IR2023★ 1 cited
Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Training
Yibin Lei, Liang Ding, Yu Cao +3
Dense retrievers have achieved impressive performance, but their demand for abundant training data limits their application scenarios. Contrastive pre-training, which constructs ps…