DPO-Shift: Shifting the Distribution of Direct Preference Optimization

Xiliang Yang, Feng Jiang, Qianen Zhang +2

Direct Preference Optimization (DPO) and its variants have become increasingly popular for aligning language models with human preferences. These methods aim to teach models to bet…