Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization

Dongwei Wang, Huanrui Yang

Quantization is a critical step to enable efficient LLM serving under limited resource. However, previous research observes that certain weights in the LLM, known as outliers, are…