Leveraging Uncertainty Estimation for Efficient LLM Routing

Tuo Zhang, Asal Mehradfar, Dimitrios Dimitriadis +1

Deploying large language models (LLMs) in edge-cloud environments requires an efficient routing strategy to balance cost and response quality. Traditional approaches prioritize eit…