1 citations · 2 across the 8 of their papers we have counts for
Showing 2024 · cs.LGShow all
2 papers · 2 filters
cs.LG2024
On the Role of Discrete Representation in Sparse Mixture of Experts
Giang Do, Kha Pham, Hung Le +1
Sparse mixture of experts (SMoE) is an effective solution for scaling up model capacity without increasing the computational costs. A crucial component of SMoE is the router, respo…
cs.LG2024★ 1 cited
CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition
Quang Pham, Giang Do, Huy Nguyen +8
Sparse mixture of experts (SMoE) offers an appealing solution to scale up the model complexity beyond the mean of increasing the network's depth or width. However, effective traini…