12 citations · 22 across the 13 of their papers we have counts for
4 papers · 1 filter
Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling
Cunchen Hu, Liangliang Xu, Tian Liu +9
Large language model (LLM) serving spans diverse applications with stringent service-level objectives (SLOs), often requiring GPUs to run at maximum frequencies and increasing ener…
LUMEN: Coordinated Failure Recovery for Distributed LLM Serving
Zhang Cao, Shujie Han, Juncheng Zhang +3
Modern large language model (LLM) serving clusters distribute inference requests across multiple worker processes on different GPUs, but failures are prevalent at scale. When a wor…
Leveraging I/O Stalls for Efficient Scheduling in ANNS
Juncheng Zhang, Yuanming Ren, Yongkun Li +1
Disk-based graph indexes for approximate nearest neighbor search (ANNS) must serve latency-sensitive queries and throughput-demanding updates concurrently. We observe that over 40%…
Varuna: Enabling Failure-Type Aware RDMA Failover
Xiaoyang Wang, Yongkun Li, Lulu Yao +7
RDMA link failures can render connections temporarily unavailable, causing both performance degradation and significant recovery overhead. To tolerate such failures, production dat…