Predicting Quantization Price for Selecting PTQ Configurations Before Deployment

Junbin Qiu, Jian Mu, Weitong Zhang +1

Weight-space post-training quantization (PTQ) must choose finite formats, granularities, quantizer families, transformations, and bits before the completed quantized model reveals…