Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts

Taneesh Gupta, Rahul Madhavan, Xuchao Zhang +3

Direct Preference Optimization (DPO) has become a popular approach for aligning language models using pairwise preferences. However, in practical post-training pipelines, on-policy…