Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts
Venmugil Elango, Nidhi Bhatia, Roger Waleffe +13
Mixture of Experts (MoEs) have become a central component of many state-of-the-art open-source and proprietary large language models. Despite their widespread adoption, it remains…
cs.LG2025★ 8 cited
ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration
Mengting Ai, Tianxin Wei, Yifan Chen +7
Mixture-of-Experts (MoE) Transformer, the backbone architecture of multiple phenomenal language models, leverages sparsity by activating only a fraction of model parameters for eac…