GS-MoE: Training-Free MoE Compression via Groupwise Subspace Sharing and Pruning

Carson Cary, Yukai Jin, Yihao Huang +5

Mixture-of-Experts (MoE) models reduce computation through sparse activation, but storing their expert parameters remains a major memory bottleneck. We introduce GS-MoE, a training…