Do Compressed LLMs Forget Knowledge? An Experimental Study with Practical Implications

Duc N. M Hoang, Minsik Cho, Thomas Merth +2

Compressing Large Language Models (LLMs) often leads to reduced performance, especially for knowledge-intensive tasks. In this work, we dive into how compression damages LLMs' inhe…