Showing cs.DCShow all
3 papers · 1 filter
cs.DC2026
InstantInfer: Enabling Fast LLM Cold Start with Communicating Finite Automata
Yitao Yuan, Yongchao He, Shaoke Fang +1
Cold starts in large language model (LLM) inference services significantly affect user experience, yet they remain inefficient due to sequential initialization and a massive number…
cs.DC2026
A Readiness-Driven Runtime for Pipeline-Parallel Training under Runtime Variability
Ruitao Liu, Xinyang Tian, Shuo Chen +4
Pipeline parallelism is a key technique for scaling large-model training, but modern workloads exhibit runtime variability in computation and communication. Existing pipeline syste…
cs.DC2025
CascadeInfer: Length-Aware Scheduling of LLM Serving with Low Latency and Load Balancing
Yitao Yuan, Chenqi Zhao, Bohan Zhao +3
Efficiently harnessing GPU compute is critical to improving user experience and reducing operational costs in large language model (LLM) services. However, current inference engine…