3 citations · 3 across the 4 of their papers we have counts for
6 papers
REVA: Reusable Evidence View Aggregation for Context-Efficient RAG Serving
Tuan Nguyen, Qiran Hu, Banruo Liu +3
Retrieval-augmented generation (RAG) improves knowledge-intensive large language model (LLM) applications by conditioning generation on retrieved documents, but longer contexts inc…
CacheFlow: Efficient LLM Serving via Automated 3D-Parallel KV Cache Restoration
Sean Nian, Jiahao Fang, Hanzhang Shen +3
KV cache restoration is becoming a major bottleneck in long-context LLM serving, including multi-turn conversations, agentic workloads, and RAG. Existing advances largely treat res…
Dora: QoE-Aware Hybrid Parallelism for Distributed Edge AI
Jianli Jin, Ziyang Lin, Qianli Dong +5
With the proliferation of edge AI applications, satisfying user quality of experience (QoE) requirements, such as model inference latency, has become a first class objective, as th…
Single-agent or Multi-agent Systems? Why Not Both?
Mingyan Gao, Yanzi Li, Banruo Liu +4
Multi-agent systems (MAS) decompose complex tasks and delegate subtasks to different large language model (LLM) agents and tools. Prior studies have reported the superior accuracy…
JITServe: SLO-aware LLM Serving with Imprecise Request Information
Wei Zhang, Zhiyu Wu, Yi Mu +5
The integration of Large Language Models (LLMs) into applications ranging from interactive chatbots to multi-agent systems has introduced a wide spectrum of service-level objective…
Compass: SLO-aware Query Planner for Compound AI Serving at Scale
Banruo Liu, Wei-Yu Lin, Minghao Fang +2
The rise of compound AI serving that integrates multiple operators in a pipeline enables end-user applications such as generative AI-powered meeting companions, autonomous driving,…