Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models

Hancheol Park, Geonho Lee, Tairen Piao +1

Mixture-of-Experts (MoE) models scale foundation models efficiently by activating only a subset of experts for each token, but their large number of expert parameters still makes q…