2 papers
cs.AI2025
Rethinking DPO: The Role of Rejected Responses in Preference Misalignment
Jay Hyeon Cho, JunHyeok Oh, Myunsoo Kim +1
Direct Preference Optimization (DPO) is a simple and efficient framework that has attracted substantial attention. However, it often struggles to meet its primary objectives -- inc…
cs.LG2024
VPO: Leveraging the Number of Votes in Preference Optimization
Jae Hyeon Cho, Minkyung Park, Byung-Jun Lee
Direct Preference Optimization (DPO) trains a language model using human preference data, bypassing the explicit reward modeling phase of Reinforcement Learning from Human Feedback…