Router Prior Bias: Preserving Base Routing Structure in MoE Post-Training

Jaedeok Lee, Keonwoo Kim, Dongyoon Han +3

Mixture-of-Experts (MoE) pretraining relies on an auxiliary load-balancing loss (LBL) to drive per-expert utilization toward uniformity. Post-training inherits a different situatio…