MixQuant: Adaptive Mixed-Precision Quantization for Large Language Models

Ashitabh Misra, Madhav Agrawal, Arham Jain +1

Mixed-precision quantization improves the accuracy of post-training quantization by allocating higher bitwidths to sensitive layers, but existing methods solve the allocation for a…