SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMs

Patrik Czakó, Gábor Kertész, Sándor Szénási

We present SmoothRot, a novel post-training quantization technique to enhance the efficiency of 4-bit quantization in Large Language Models (LLMs). SmoothRot addresses the critical…