3 citations · 4 across the 6 of their papers we have counts for
1 paper · 1 filter
Xuan-Phi Nguyen, Shrey Pandit, Austin Xu +2
Mixture-of-Experts (MoE) models are typically pre-trained with explicit load-balancing constraints to ensure statistically balanced expert routing. Despite this, we observe that ev…