More Than Catastrophic Forgetting: Integrating General Capabilities For Domain-Specific LLMs

Chengyuan Liu, Yangyang Kang, Shihang Wang +5

The performance on general tasks decreases after Large Language Models (LLMs) are fine-tuned on domain-specific tasks, the phenomenon is known as Catastrophic Forgetting (CF). Howe…