CoLLM: Continuous Adaptation for SLO-Aware LLM Serving on Shared GPU Clusters

Shaoyuan Huang, Yunfeng Zhao, Na Yan +5

As Large Language Models (LLMs) are increasingly adopted in edge intelligence to power domain-specific applications and personalized services, the quality and efficiency of the LLM…