activity
20232025
most citedCompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition

1 citations · 3 across the 8 of their papers we have counts for

collaborators
Showing 2023Show all

1 paper · 1 filter

cs.LG2023

HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts

Giang Do, Khiem Le, Quang Pham +7

By routing input tokens to only a few split experts, Sparse Mixture-of-Experts has enabled efficient training of large language models. Recent findings suggest that fixing the rout…