3 citations · 3 across the 4 of their papers we have counts for
Showing cs.DCShow all
3 papers · 1 filter
cs.DC2026
CacheFlow: Efficient LLM Serving via Automated 3D-Parallel KV Cache Restoration
Sean Nian, Jiahao Fang, Hanzhang Shen +3
KV cache restoration is becoming a major bottleneck in long-context LLM serving, including multi-turn conversations, agentic workloads, and RAG. Existing advances largely treat res…
cs.DC2025
Dora: QoE-Aware Hybrid Parallelism for Distributed Edge AI
Jianli Jin, Ziyang Lin, Qianli Dong +5
With the proliferation of edge AI applications, satisfying user quality of experience (QoE) requirements, such as model inference latency, has become a first class objective, as th…
cs.DC2025
JITServe: SLO-aware LLM Serving with Imprecise Request Information
Wei Zhang, Zhiyu Wu, Yi Mu +5
The integration of Large Language Models (LLMs) into applications ranging from interactive chatbots to multi-agent systems has introduced a wide spectrum of service-level objective…