Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspective

Junpeng Zhang, Lei Cheng, Guoxi Zhang +3

This paper explores a scientific question in supervised fine-tuning (SFT): why SFT is broadly effective for small-scale deep neural networks, yet can produce inconsistent or even d…