HERO-MoE: Historical Expert Routing with Scale-Preserving Fusion

Junxiang Qiu, Zhengsu Chen, Xinting Hu +6

Mixture-of-Experts (MoE) architectures have become a standard way to scale model capacity while keeping computation sparse, yet routing remains a key determinant of MoE quality and…