5 papers
DeepShare: Assurance-Driven Deep Learning Job Scheduling for Multi-Tenant Clusters
Jinghao Wang, Yihang Zhou, Xiao Zhou +5
Multi-tenant GPU clusters frequently remain underutilized even when tenants experience long queueing delays, because quota control, queue ordering, preemption, and GPU sharing are…
Latency-Aware Orchestration for Multi-Agent LLM Workflows on Heterogeneous GPUs
Jinghao Wang, Yifeng Zhang, Xiao Zhou +7
Concurrent multi-agent workflows expose future dependencies and serving-state requirements while running on heterogeneous GPU pools with time-varying load, model residency, and res…
ElastiCo: Elastic Configuration and Interference-Aware Orchestration for GPU Clusters
Jinghao Wang, Yihang Zhou, Xiaoyang Sun +5
Modern GPU clusters must simultaneously serve deep learning training and offline large language model inference workloads, yet existing schedulers treat these as isolated resource…
CrossPool: Efficient Multi-LLM Serving for Cold MoE Models through KV-Cache and Weight Disaggregation
Zhuoren Ye, Tianyu Wo, Dinghao Xue +4
Emerging LLM services increasingly host many sparse MoE models, yet most models receive sparse requests and remain cold. This creates a GPU memory problem: model weights are stable…
Maestro: Workload-Aware Cross-Cluster Scheduling for LLM-Based Multi-Agent Systems
Jinghao Wang, Xiao Zhou, Xiaoyang Sun +6
Large Language Model based Multi-Agent Systems (LLM-MAS) have emerged as a powerful paradigm for tackling complex tasks by breaking them into collaborative workflows of specialized…