3 papers
cs.LG2025
What Matters in Data for DPO?
Yu Pan, Zhongze Cai, Guanting Chen +2
Direct Preference Optimization (DPO) has emerged as a simple and effective approach for aligning large language models (LLMs) with human preferences, bypassing the need for a learn…
cs.LG2025
Understanding the Impact of Sampling Quality in Direct Preference Optimization
Kyung Rok Kim, Yumo Bai, Chonghuan Wang +1
We study how data of higher quality can be leveraged to improve performance in Direct Preference Optimization (DPO), aiming to understand its impact on DPO training dynamics. Our a…
stat.ML2025
Collaborative Prediction: To Join or To Disjoin Datasets
Kyung Rok Kim, Yansong Wang, Xiaocheng Li +1
With the recent rise of generative Artificial Intelligence (AI), the need of selecting high-quality dataset to improve machine learning models has garnered increasing attention. Ho…