3 papers
cs.LG2026
PeekKV: The First Content Token Already Knows Which Heads Matter
Minyoung Go, Yongjun Park
The size of the KV cache grows linearly with context length, making it a major memory bottleneck in LLM serving. Motivated by recent progress in identifying the functional roles of…
cs.DC2026
PIM-SHERPA: Software Method for On-device LLM Inference by Resolving PIM Memory Attribute and Layout Inconsistencies
Sunjung Lee, Sanghoon Cha, Hyeonsu Kim +8
On-device deployments of large language models (LLMs) are rapidly proliferating across mobile and edge platforms. LLM inference comprises a compute-intensive prefill phase and a me…
cs.LG2025
FlexiQ: Adaptive Mixed-Precision Quantization for Latency/Accuracy Trade-Offs in Deep Neural Networks
Jaemin Kim, Hongjun Um, Sungkyun Kim +2
Neural networks commonly execute on hardware accelerators such as NPUs and GPUs for their size and computation overhead. These accelerators are costly and it is hard to scale their…