Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
GS-MoE: Training-Free MoE Compression via Groupwise Subspace Sharing and Pruning
Carson Cary, Yukai Jin, Yihao Huang +5
Mixture-of-Experts (MoE) models reduce computation through sparse activation, but storing their expert parameters remains a major memory bottleneck. We introduce GS-MoE, a training…
cs.LG2025
ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning
Shangqian Gao, Ting Hua, Reza Shirkavand +10
Large Language Models (LLMs) have demonstrated remarkable abilities in tackling a wide range of complex tasks. However, their huge computational and memory costs raise significant…