FPTQ: Fine-grained Post-Training Quantization for Large Language Models

Qingyuan Li, Yifan Zhang, Liang Li +6

In the era of large-scale language models, the substantial parameter size poses significant challenges for deployment. Being a prevalent compression technique, quantization has eme…