RLCascadeRouter: Quality-Estimator-Free Cascade Routing via Reinforcement Learning

Shihong Huang, Shengjie Wang, Hong Ma +1

The growing ecosystem of large language models (LLMs) offers huge potential to optimize performance-cost trade-offs. However, their heterogeneous capabilities and inference costs m…