5 papers
LUMEN: Coordinated Failure Recovery for Distributed LLM Serving
Zhang Cao, Shujie Han, Juncheng Zhang +3
Modern large language model (LLM) serving clusters distribute inference requests across multiple worker processes on different GPUs, but failures are prevalent at scale. When a wor…
Leveraging I/O Stalls for Efficient Scheduling in ANNS
Juncheng Zhang, Yuanming Ren, Yongkun Li +1
Disk-based graph indexes for approximate nearest neighbor search (ANNS) must serve latency-sensitive queries and throughput-demanding updates concurrently. We observe that over 40%…
TierCheck: Tiered Checkpointing for Fault Tolerance in Large Language Model Training
Shujie Han, Feng Jiang, Patrick P. C. Lee +5
Large Language Model (LLM) training is frequently interrupted by a heterogeneous spectrum of failures, from common GPU crashes to catastrophic cluster-wide outages. Existing checkp…
FluxMoE: Decoupling Expert Residency for High-Performance MoE Serving
Qingxiu Liu, Cyril Y. He, Hanser Jiang +3
Mixture-of-Experts (MoE) models have become a dominant paradigm for scaling large language models, but their rapidly growing parameter sizes introduce a fundamental inefficiency du…
The Design and Implementation of a High-Performance Log-Structured RAID System for ZNS SSDs
Jinhong Li, Yiyang Geng, Qiuping Wang +2
Zoned Namespace (ZNS) defines a new abstraction for host software to flexibly manage storage in flash-based SSDs as append-only zones. It also provides a Zone Append primitive to f…