1 citations · 1 across the 3 of their papers we have counts for
Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
PEEK: Predictive Queue-Informed KV Cache Management for LLM Serving
Bing Xie, Zhipeng Wang, Masahiro Tanaka +2
We present PEEK, a lightweight scheduling and eviction framework for both online (streaming) and offline (batch) LLM serving; this paper focuses on the online regime. PEEK maintain…
cs.DC2024★ 1 cited
FastPersist: Accelerating Model Checkpointing in Deep Learning
Guanhua Wang, Olatunji Ruwase, Bing Xie +1
Model checkpoints are critical Deep Learning (DL) artifacts that enable fault tolerance for training and downstream applications, such as inference. However, writing checkpoints to…