When Quantization Affects Confidence of Large Language Models?

Irina Proskurina, Luc Brun, Guillaume Metzler +1

Recent studies introduced effective compression techniques for Large Language Models (LLMs) via post-training quantization or low-bit weight representation. Although quantized weig…