2 papers
cs.DC2026
From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving
Jie Li, Tongyang Wang, Yong Chen
The key-value (KV) cache has become a first-order memory object in LLM serving rather than a temporary per-request tensor. This survey classifies more than thirty KV-management sys…
cs.DC2023
Analyzing Resource Utilization in an HPC System: A Case Study of NERSC Perlmutter
Jie Li, George Michelogiannakis, Brandon Cook +2
Resource demands of HPC applications vary significantly. However, it is common for HPC systems to primarily assign resources on a per-node basis to prevent interference from co-loc…