Learning Style, Forgetting Semantics: A Case Study of SFT and RFT on Classification Tasks

Haodong Liang, Yanhao Jin, Krishnakumar Balasubramanian +1

Why does supervised fine-tuning (SFT) lead to more forgetting than reinforcement fine-tuning (RFT), even when all teacher demonstrations are semantically correct? We study this que…