Showing cs.CLShow all
3 papers · 1 filter
cs.CL2026
Rethinking Sparse Mixture of Experts from a Unified Perspective
Giang Do, Hung Le, Truyen Tran
Sparse Mixture of Experts (SMoE) models scale the capacity of models while maintaining constant computational overhead. SMoE methods fall into two categories: Token Choice, which r…
cs.CL2026
Do Domain-specific Experts exist in MoE-based LLMs?
Giang Do, Hung Le, Truyen Tran
In the era of Large Language Models (LLMs), the Mixture of Experts (MoE) architecture has emerged as an effective approach for training extremely large models with improved computa…
cs.CL2025
S2MoE: Robust Sparse Mixture of Experts via Stochastic Learning
Giang Do, Hung Le, Truyen Tran
Sparse Mixture of Experts (SMoE) enables efficient training of large language models by routing input tokens to a select number of experts. However, training SMoE remains challengi…