2 papers
cs.DC2026
Beyond Greedy Chunking: SLO-Aware Sliding-Window Scheduling for LLM Inference
Yuansheng Chen, Yue Zhang, Xuan Mo +2
With the rapid growth of interactive applications in large language model (LLM) online services, maintaining high system throughput while ensuring user-perceived latency has become…
cs.DC2025
A Predictive and Synergistic Two-Layer Scheduling Framework for LLM Serving
Yue Zhang, Yuansheng Chen, Xuan Mo +3
LLM inference serving typically scales out with a two-tier architecture: a cluster router distributes requests to multiple inference engines, each of which then in turn performs it…