Efficient Training-Free Online Routing for High-Volume Multi-LLM Serving

Fangzhou Wu, Sandeep Silwal

Increasing demand for Large Language Models (LLMs) services imposes substantial deployment and computation costs on providers. LLM routing offers a cost-efficient solution by direc…