Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Experts

Jin Yang, Qiong Wu, Zhiying Feng +3

Large Language Models (LLMs) have demonstrated remarkable capabilities, leading to a significant increase in user demand for LLM services. However, cloud-based LLM services often s…