Understanding the Impact of Sampling Quality in Direct Preference Optimization

Kyung Rok Kim, Yumo Bai, Chonghuan Wang +1

We study how data of higher quality can be leveraged to improve performance in Direct Preference Optimization (DPO), aiming to understand its impact on DPO training dynamics. Our a…