most citedCompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition

1 citations · 1 across the 1 of their papers we have counts for

collaborators

1 paper

cs.LG2024★ 1 cited

CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition

Quang Pham, Giang Do, Huy Nguyen +8

Sparse mixture of experts (SMoE) offers an appealing solution to scale up the model complexity beyond the mean of increasing the network's depth or width. However, effective traini…