WarmServe: Enabling One-for-Many GPU Prewarming for Multi-LLM Serving

Chiheng Lou, Sheng Qi, Rui Kang +5

Deploying multiple models within shared GPU clusters is a key strategy to improve resource efficiency in large language model (LLM) serving. Existing multi-LLM serving systems impr…