5 papers
Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling
Cunchen Hu, Liangliang Xu, Tian Liu +9
Large language model (LLM) serving spans diverse applications with stringent service-level objectives (SLOs), often requiring GPUs to run at maximum frequencies and increasing ener…
LUMEN: Coordinated Failure Recovery for Distributed LLM Serving
Zhang Cao, Shujie Han, Juncheng Zhang +3
Modern large language model (LLM) serving clusters distribute inference requests across multiple worker processes on different GPUs, but failures are prevalent at scale. When a wor…
Leveraging I/O Stalls for Efficient Scheduling in ANNS
Juncheng Zhang, Yuanming Ren, Yongkun Li +1
Disk-based graph indexes for approximate nearest neighbor search (ANNS) must serve latency-sensitive queries and throughput-demanding updates concurrently. We observe that over 40%…
Varuna: Enabling Failure-Type Aware RDMA Failover
Xiaoyang Wang, Yongkun Li, Lulu Yao +7
RDMA link failures can render connections temporarily unavailable, causing both performance degradation and significant recovery overhead. To tolerate such failures, production dat…
FOCUS: Boosting Schema-aware Access for KV Stores via Hierarchical Data Management
Zhen Liu, Wenzhe Zhu, Yongkun Li +1
Persistent key-value (KV) stores are critical infrastructure for data-intensive applications. Leveraging high-performance Non-Volatile Memory (NVM) to enhance KV stores has gained…