Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
CommVQ: Commutative Vector Quantization for KV Cache Compression
Junyan Li, Yang Zhang, Muhammad Yusuf Hassan +8
Large Language Models (LLMs) are increasingly used in applications requiring long context lengths, but the key-value (KV) cache often becomes a memory bottleneck on GPUs as context…
cs.CL2025
Steering LLM Thinking with Budget Guidance
Junyan Li, Wenshuo Zhao, Yang Zhang +1
Recent deep-thinking large language models often reason extensively to improve performance, but such lengthy reasoning is not always desirable, as it incurs excessive inference cos…