When is Routing Meaningful? Diversity and Robustness in Language Model Societies

Fantine Huot, Michael Kaisers, Mirella Lapata

Routing policies for multi-model systems are evaluated almost exclusively on task accuracy and inference cost. We argue that two properties, orthogonal to performance, determine wh…