SFT-as-Context Mitigates Forgetting in Supervised Fine-Tuning

Kenan Tang, Andong Hua, Chengxuan Qian +2

Supervised fine-tuning (SFT) equips large language models (LLMs) with specialized capabilities, but often comes at the cost of forgetting the general capabilities of their parent m…