Integer Scale: A Free Lunch for Faster Fine-grained Quantization of LLMs

Qingyuan Li, Ran Meng, Yiduo Li +5

We introduce Integer Scale, a novel post-training quantization scheme for large language models that effectively resolves the inference bottleneck in current fine-grained quantizat…