Efficiently Editing Mixture-of-Experts Models with Compressed Experts

Yifei He, Yang Liu, Chen Liang +1

Mixture-of-Experts (MoE) models have become a key approach for scaling large language models efficiently by activating only a subset of experts during training and inference. Typic…