An Empirical Study of SFT-DPO Interaction and Parameterization in Small Language Models

Yuming Feng, Christy Yang

Direct Preference Optimization (DPO) is widely used after supervised fine-tuning (SFT) to align language models, yet empirical behavior under small backbones and modest data is und…