3 papers
cs.LG2025
What Matters in Data for DPO?
Yu Pan, Zhongze Cai, Guanting Chen +2
Direct Preference Optimization (DPO) has emerged as a simple and effective approach for aligning large language models (LLMs) with human preferences, bypassing the need for a learn…
stat.ML2025
Collaborative Prediction: To Join or To Disjoin Datasets
Kyung Rok Kim, Yansong Wang, Xiaocheng Li +1
With the recent rise of generative Artificial Intelligence (AI), the need of selecting high-quality dataset to improve machine learning models has garnered increasing attention. Ho…
cs.LG2025
Understanding the Impact of Sampling Quality in Direct Preference Optimization
Kyung Rok Kim, Yumo Bai, Chonghuan Wang +1
We study how data of higher quality can be leveraged to improve performance in Direct Preference Optimization (DPO), aiming to understand its impact on DPO training dynamics. Our a…