Adaptive Inverted-Index Routing for Granular Mixtures-of-Experts

Klaus-Rudolf Kladny, Maximilian Mordig, Bernhard Schölkopf +1

Sparse Mixture-of-experts (MoE) models enable scalable transformer architectures by activating only a subset of experts per token. Recent evidence suggests that performance improve…