Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning

Weijieying Ren, Xinlong Li, Lei Wang +2

Existing research has shown that large language models (LLMs) exhibit remarkable performance in language understanding and generation. However, when LLMs are continuously fine-tune…