Rethinking the Outlier Distribution in Large Language Models: An In-depth Study

Rahul Raman, Khushi Sharma, Sai Qian Zhang

Investigating outliers in large language models (LLMs) is crucial due to their significant impact on various aspects of LLM performance, including quantization and compression. Out…