UELLM: A Unified and Efficient Approach for LLM Inference Serving

Yiyuan He, Minxian Xu, Jingfeng Wu +3

In the context of Machine Learning as a Service (MLaaS) clouds, the extensive use of Large Language Models (LLMs) often requires efficient management of significant query loads. Wh…