The Routing Plateau: Understanding and Breaking the Accuracy Limits of LLM Routers

Yifan Lu, Qiyue Zhang, Shenrun Zhang +4

LLM routing has become a popular approach to improve the cost-quality trade-off of LLM services by dynamically selecting a model for each query. Recent work has explored a broad ra…