6 citations · 6 across the 4 of their papers we have counts for
Showing cs.DCShow all
3 papers · 1 filter
cs.DC2026
CacheFlow: Efficient LLM Serving via Automated 3D-Parallel KV Cache Restoration
Sean Nian, Jiahao Fang, Hanzhang Shen +3
KV cache restoration is becoming a major bottleneck in long-context LLM serving, including multi-turn conversations, agentic workloads, and RAG. Existing advances largely treat res…
cs.DC2025
JITServe: SLO-aware LLM Serving with Imprecise Request Information
Wei Zhang, Zhiyu Wu, Yi Mu +5
The integration of Large Language Models (LLMs) into applications ranging from interactive chatbots to multi-agent systems has introduced a wide spectrum of service-level objective…
cs.DC2024★ 6 cited
Andes: Defining and Enhancing Quality-of-Experience in LLM-Based Text Streaming Services
Jiachen Liu, Jae-Won Chung, Zhiyu Wu +3
Large language models (LLMs) are now at the core of conversational AI services such as real-time translation and chatbots, which provide live user interaction by incrementally stre…