QAQ: Quality Adaptive Quantization for LLM KV Cache

Shichen Dong, Wen Cheng, Jiayu Qin +1

The emergence of LLMs has ignited a fresh surge of breakthroughs in NLP applications, particularly in domains such as question-answering systems and text generation. As the need fo…