Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting

Fei Ding, Baiqiao Wang

Supervised Fine-Tuning (SFT) is a critical step for enhancing the instruction-following capabilities of Large Language Models (LLMs) and adapting them to specialized domains. Howev…