Cosine-Similarity Routing with Semantic Anchors for Interpretable Mixture-of-Experts Language Models

Ivan Ternovtsii, Yurii Bilak

Mixture-of-Experts (MoE) models improve efficiency through sparse activation, but their learned gating functions provide limited insight into routing decisions. This work introduce…