Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs

Davide Paglieri, Saurabh Dash, Tim Rocktäschel +1

Post-Training Quantization (PTQ) enhances the efficiency of Large Language Models (LLMs) by enabling faster operation and compatibility with more accessible hardware through reduce…