Analyzing the Effect of Noise in LLM Fine-tuning

Lingfang Li, Procheta Sen

Fine-tuning is the dominant paradigm for adapting pretrained large language models (LLMs) to downstream NLP tasks. In practice, fine-tuning datasets may contain various forms of no…