Target-Aware Calibration Data Selection for Preserving Uncertainty in Quantized Language Models

Zhen Yang, Sizai Hou, Kaiwen Zheng +4

Quantization is widely used to deploy large language models, but its effect on uncertainty behavior, such as confidence, margins, and abstention, is rarely treated as a primary obj…