Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals

Yamato Narita, Issei Sato

Post-training weight-activation quantization reduces the memory and inference costs of large language models, but aggressive W4A4 quantization remains difficult because activation…