1 citations · 1 across the 2 of their papers we have counts for
3 papers
cs.DC2026
PEEK: Predictive Queue-Informed KV Cache Management for LLM Serving
Bing Xie, Zhipeng Wang, Masahiro Tanaka +1
We present PEEK, a lightweight scheduling and eviction framework for both online (streaming) and offline (batch) LLM serving; this paper focuses on the online regime. PEEK maintain…
cs.DC2026★ 1 cited
DeepCompile: A Compiler-Driven Approach to Optimizing Distributed Deep Learning Training
Masahiro Tanaka, Du Li, Umesh Chand +3
The rapid growth of deep learning models has increased the demand for efficient distributed training strategies. Fully sharded approaches like ZeRO-3 and FSDP partition model param…
cs.CL2025
Mitigating KV Cache Competition to Enhance User Experience in LLM Inference
Haiying Shen, Tanmoy Sen, Masahiro Tanaka
In Large Language Model (LLM) serving, the KV-cache (KVC) bottleneck causes high tail Time-to-First-Token (TTFT) and Time-Between-Tokens (TBT), impairing user experience, particula…