KVLinC : KV Cache Quantization with Hadamard Rotation and Linear Correction

Utkarsh Saxena, Kaushik Roy

Quantizing the key-value (KV) cache is a promising strategy for improving the inference efficiency of large language models (LLMs). However, aggressive quantization to very low pre…