Geometric Metrics for MoE Specialization: From Fisher Information to Early Failure Detection

Dongxin Guo, Jikun Wu, Siu Ming Yiu

Expert specialization is fundamental to Mixture-of-Experts (MoE) model success, yet existing metrics (cosine similarity, routing entropy) lack theoretical grounding and yield incon…