DeepServe: Serverless Large Language Model Serving at Scale

Junhao Hu, Jiang Xu, Zhixia Liu +18

In this paper, we propose DEEPSERVE, a scalable and serverless AI platform designed to efficiently serve large language models (LLMs) at scale in cloud environments. DEEPSERVE addr…