Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning

Yang Zhao, Li Du, Xiao Ding +5

Through pretraining on a corpus with various sources, Large Language Models (LLMs) have gained impressive performance. However, the impact of each component of the pretraining corp…