λScale: Enabling Fast Scaling for Serverless Large Language Model Inference

Minchen Yu, Rui Yang, Chaobo Jia +9

Serverless computing has emerged as a compelling solution for cloud-based model inference. However, as modern large language models (LLMs) continue to grow in size, existing server…