3 papers
cs.LG2026
KVSculpt: KV Cache Compression as Distillation
Bo Jiang, Sian Jin
KV cache compression is critical for efficient long-context LLM inference. Approaches that reduce the per-pair footprint -- quantization and low-rank decomposition -- are orthogona…
cs.DC2026
PackKV: Reducing KV Cache Memory Footprint through LLM-Aware Lossy Compression
Bo Jiang, Taolue Yang, Youyuan Liu +3
Transformer-based large language models (LLMs) have demonstrated remarkable potential across a wide range of practical applications. However, long-context inference remains a signi…
cs.DC2025
KVComp: A High-Performance, LLM-Aware, Lossy Compression Framework for KV Cache
Bo Jiang, Taolue Yang, Youyuan Liu +3
Transformer-based large language models (LLMs) demonstrate impressive potential in various practical applications. However, long context inference poses a significant challenge due…