decoupleQ: Towards 2-bit Post-Training Uniform Quantization via decoupling Parameters into Integer and Floating Points

Yi Guo, Fanliu Kong, Xiaoyang Li +6

Quantization emerges as one of the most promising compression technologies for deploying efficient large models for various real time application in recent years. Considering that…