Adaptive Request Scheduling for CodeLLM Serving with SLA Guarantees

Shi Chang, Boyuan Chen, Kishanthan Thangarajah +2

Code Large Language Models (CodeLLMs) are increasingly integrated into modern software development workflows, yet efficiently serving them in resource-constrained, self-hosted envi…