Activation Sensitivity as a Unifying Principle for Post-Training Quantization

Abstract

Post-training quantization (PTQ) methods for large language models compress pretrained weights to low bit-widths without retraining by optimizing layer-local proxy objectives. Activation-aware scaling methods such as AWQ protect input channels with large activation norms, whereas second-order error-compensation methods such as GPTQ redistribute rounding error using the inverse of the unweighted input covariance matrix. We show that both paradigms -- along with diagonal Fisher quantization, block reconstruction, and Hadamard rotation methods -- are special cases of minimizing a single loss-derived quadratic form governed by gradient-weighted activation sensitivity. From a Gauss-Newton expansion of the task loss at a linear layer $Y=X{W}^{\top }$, we prove that under isotropic output curvature the expected excess loss from weight perturbation $\Delta W$ reduces to $\frac{1}{2d_{\mathrm{out}}}\mathrm{Tr}(\Delta W{H}_D\Delta W^{\top })$, where $H_D=\frac{1}{n}{X}^{\top }DX$ weights each calibration token's outer product by its squared output-gradient norm $D_{tt}=\Vert {\nabla }_{y_t}\mathcal{L}{\Vert }_2^2$. Within this framework, AWQ corresponds to a diagonal approximation of $H_D$ with uniform token weights ($D=I_n$) optimized via channel scaling, while GPTQ corresponds to Cholesky error compensation on $H_D$ under $D=I_n$. We derive closed-form optimal scaling exponents for diagonal sensitivity protection, formulate sensitivity-weighted Cholesky quantization (SAQ-GPTQ) on $H_D$, and prove bounds on token-gradient heterogeneity bias, cross-layer error amplification across residual blocks, and finite-sample calibration concentration. Controlled numerical simulations confirm that under token-gradient heterogeneity, replacing $X^{\top }X$ with $X^{\top }DX$ preserves $>99\%$ top-channel saliency recall and reduces INT3/INT4 excess loss by roughly $3\times$ over unweighted GPTQ.