Linear Preference Optimization: Decoupled Gradient Control via Absolute Regularization

Rui Wang, Qianguo Sun, Chao Song +4

DPO (Direct Preference Optimization) has become a widely used offline preference optimization algorithm due to its simplicity and training stability. However, DPO is prone to overf…