Beyond Query Memorization: Large Language Model Routing with Query Decomposition and Historical Matching

Bo Lv, Jingbo Sun

Optimizing the trade-off among predictive performance and computational cost is a central focus in the deployment of Large Language Models (LLMs). Current routing methods primarily…