3 papers
cs.DC2026
Beyond Greedy Chunking: SLO-Aware Sliding-Window Scheduling for LLM Inference
Yuansheng Chen, Yue Zhang, Xuan Mo +2
With the rapid growth of interactive applications in large language model (LLM) online services, maintaining high system throughput while ensuring user-perceived latency has become…
cs.DC2025
A Predictive and Synergistic Two-Layer Scheduling Framework for LLM Serving
Yue Zhang, Yuansheng Chen, Xuan Mo +3
LLM inference serving typically scales out with a two-tier architecture: a cluster router distributes requests to multiple inference engines, each of which then in turn performs it…
cs.DC2025
MaaSO: SLO-aware Orchestration of Heterogeneous Model Instances for MaaS
Mo Xuan, Zhang yue, Wu Weigang
Model-as-a-Service (MaaS) platforms face diverse Service Level Objective (SLO) requirements stemming from various large language model (LLM) applications, manifested in contextual…