Guarded Query Routing for Large Language Models

Richard Šléher, William Brach, Tibor Sloboda +2

Query routing, the task to route user queries to different large language model (LLM) endpoints, can be considered as a text classification problem. However, out-of-distribution qu…