$\xi$-DPO: Direct Preference Optimization via Ratio Reward Margin

Zhengyuan Fan, Zhonghua Wu, Yuxuan Du +1

Reference-free preference optimization has emerged as an efficient alternative to reinforcement learning from human feedback, with Simple Preference Optimization(SimPO) demonstrati…