FidelityQ: Preserving Predictive Fidelity in Ultra-Low-Bit Language Model Quantization

Yujie Chen, Weiyan Zheng, Yiwu Yao +7

Ultra-low-bit scalar quantization offers a regular arithmetic structure for efficient deployment of large language models, but preserving predictive behavior at extreme precision r…