2 papers
cs.LG2026
Distance-KV: Exploiting Relative Distance for Efficient Long-Context Inference
Xianpeng Shang, Canbin Huang, Jiang Li +4
The memory usage and decoding latency of LLM inference grow rapidly with context length. To reduce these costs, key-value (KV) cache compression methods selectively retain cached s…
cs.CL2026
Training-Inference Consistent Segmented Execution for Long-Context LLMs
Xianpeng Shang, Jiang Li, Zehua Duo +2
Transformer-based large language models face severe scalability challenges in long-context generation due to the computational and memory costs of full-context attention. Under pra…