BAQ: Efficient Bit Allocation Quantization for Large Language Models

Chao Zhang, Li Wang, Samson Lasaulce +1

Post-training model quantization is a widely adopted technique for reducing the memory and computational costs of large language models (LLMs). However, most existing methods rely…