3 papers
cs.AR2026
Adaptive Multi-Objective Tiered Storage Configuration for KV Cache in LLM Service
Xianzhe Zheng, Zhengheng Wang, Ruiyan Ma +17
The memory-for-computation paradigm of KV caching is essential for accelerating large language model (LLM) inference service, but limited GPU high-bandwidth memory (HBM) capacity m…
cs.CV2025
VLCache: Computing 2% Vision Tokens and Reusing 98% for Vision-Language Inference
Shengling Qin, Hao Yu, Chenxin Wu +10
This paper presents VLCache, a cache reuse framework that exploits both Key-Value (KV) cache and encoder cache from prior multimodal inputs to eliminate costly recomputation when t…
cs.DB2023
Async-fork: Mitigating Query Latency Spikes Incurred by the Fork-based Snapshot Mechanism from the OS Level
Pu Pang, Gang Deng, Kaihao Bai +12
In-memory key-value stores (IMKVSes) serve many online applications because of their efficiency. To support data backup, popular industrial IMKVSes periodically take a point-in-tim…