QET: Enhancing Quantized LLM Parameters and KV cache Compression through Element Substitution and Residual Clustering

Yanshu Wang, Wang Li, Zhaoqian Yao +1

The matrix quantization entails representing matrix elements in a more space-efficient form to reduce storage usage, with dequantization restoring the original matrix for use. We f…