2 papers
cs.LG2026
Evidence for Shared Routing Geometry and Dynamics in Sparse Mixture-of-Experts
Kirill Labzin, Stepan Kulibaba, Artem Dzhalilov +1
Sparse mixture-of-experts (MoE) models use an independently parameterized router at each sparse layer to select experts for every token. Prior work has shown that routing decisions…
cs.LG2026
SDG-MoE: Signed Debate Graph Mixture-of-Experts
Stepan Kulibaba, Kirill Labzin, Artem Dzhalilov +4
Sparse MoE models achieve a good balance between capacity and compute by routing each token to a small subset of experts. However, in most MoE architectures, once a token is routed…