2 papers
cs.LG2026
HEAPr: Hessian-based Efficient Atomic Expert Pruning in Output Space
Ke Li, Zheng Yang, Zhongbin Zhou +3
Mixture-of-Experts (MoE) architectures in large language models (LLMs) deliver exceptional performance and reduced inference costs compared to dense LLMs. However, their large para…
cs.CL2026
HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference
Zhiyuan Shi, Qibo Qiu, Feng Xue +5
The linear memory growth of the KV cache poses a significant bottleneck for LLM inference in long-context tasks. Existing static compression methods often fail to preserve globally…