3 papers
cs.AR2026
Heterogeneous LLM Serving with General-Purpose Processing-Near-Memory for Retrieval-Based Sparse Attention
Hyungkyu Ham, Junhyeong Bae, Seungheon Lee +2
This paper presents a heterogeneous decode-phase serving system that relocates the KV cache out of GPU memory, motivated by the retrieval-based sparse attention that recent frontie…
cs.AI2026
OrbitFlow: SLO-Aware Long-Context LLM Serving with Fine-Grained KV Cache Reconfiguration
Xinyue Ma, Heelim Hong, Taegeon Um +4
Serving long-context LLMs is challenging because request lengths and batch composition vary during token generation, causing the memory footprint to fluctuate significantly at runt…
cs.LG2025
REP: Resource-Efficient Prompting for Rehearsal-Free Continual Learning
Sungho Jeon, Xinyue Ma, Kwang In Kim +1
Recent rehearsal-free continual learning (CL) methods guided by prompts achieve strong performance on vision tasks with non-stationary data but remain resource-intensive, hindering…