The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs

Baha Rababah, Shahzeb Qamar, Lorenz Sparrenberg +4

Post-Training Quantization has become widely used to compress large language models to make them deployable on resource-constrained devices. However, the evaluation of quantization…