WISERouter: LLM Routing with Workload Budget Constraint

Yifei Li, Zihui Gao, Laks V. S. Lakshmanan

Large language models (LLMs) achieve impressive performance across multiple domains, but using the most capable model for every query is prohibitive at scale. LLM routing exploits…