Outlier Smoothing with Closed-Form Rotations for W4A4 Large Language Model Quantization

Jinying Xiao, Bin Ji, Shasha Li +8

Large Language Models (LLMs) quantization facilitates deploying LLMs in resource-limited settings, but existing methods that combine incompatible gradient optimization and quantiza…