3 papers
cs.DC2026
WarmServe: Enabling One-for-Many GPU Prewarming for Multi-LLM Serving
Chiheng Lou, Sheng Qi, Rui Kang +5
Deploying multiple models within shared GPU clusters is a key strategy to improve resource efficiency in large language model (LLM) serving. Existing multi-LLM serving systems impr…
cs.LG2026
LiteCache: A Query Similarity-Driven, GPU-Centric KVCache Subsystem for Efficient LLM Inference
Jiawei Yi, Ping Gong, Youhui Bai +10
During LLM inference, KVCache memory usage grows linearly with sequence length and batch size and often exceeds GPU capacity. Recent proposals offload KV states to host memory and…
cs.SE2026
IntrinTrans: LLM-based Intrinsic Code Translator for RISC-V Vector
Liutong Han, Zhiyuan Tan, Hongbin Zhang +4
The use of intrinsic functions to leverage hardware-specific capabilities is a crucial approach for optimizing library performance. Many mainstream libraries implement a large numb…