Topology-aware Preemptive Scheduling for Co-located LLM Workloads

Ping Zhang, Lei Su, Jinjie Yang +1

Hosting diverse large language model workloads in a unified resource pool through co-location is cost-effective. For example, long-running chat services generally follow diurnal tr…