Beyond the Previous Layer: Residual Predictive Structure in Sparse MoE Routing

Hao Li, Yasuyuki Tahara, Yuichi Sei

Sparse mixture-of-experts models route each token through a sequence of expert selections. We ask whether the immediately preceding selection adequately summarizes this trajectory…