Systematic Outliers in Large Language Models

Yongqi An, Xu Zhao, Tao Yu +2

Outliers have been widely observed in Large Language Models (LLMs), significantly impacting model performance and posing challenges for model compression. Understanding the functio…