Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
SlimKV: Joint Token-Feature KV Cache Compression with Reconstruction-Free Beacon Attention
Zihan Teng, Jiayu Zhao, Wentao Ren +4
Long-context LLM serving is increasingly bottlenecked by KV-cache memory, especially in resource-constrained scenarios. Among existing KV-cache compression strategies, token-wise m…
cs.LG2026
BitsMoE: Cost-Aware Bit Allocation in Spectral Space for MoE LLM Quantization
Jiayu Zhao, Zihan Teng, Minhao Fan +4
Mixture-of-Experts (MoE) large language models incur substantial memory costs due to their large expert parameter counts. Mixed-precision quantization reduces these costs by alloca…