2 citations · 3 across the 22 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
When Load-Balancing Goes Too Far: Expert Pruning in Over-Dispersed Mixture-of-Experts Models
Berkcan Kapusuzoglu, Connor Pryor, Sangwoo Cho +4
Expert pruning reduces the memory and serving cost of Mixture-of-Experts (MoE) models by removing low-importance experts identified by the router, assuming router probabilities pro…
cs.LG2025
Continual Pre-training of MoEs: How robust is your router?
Benjamin Thérien, Charles-Étienne Joseph, Zain Sarwar +7
Sparsely-activated Mixture of Experts (MoE) transformers are promising architectures for foundation models. Compared to dense transformers that require the same amount of floating-…