${\hat{I}}^2$-DPO: Direct Preference Optimization with Dynamic ${\hat{I}}^2$

Junkang Wu, Yuexiang Xie, Zhengyi Yang +5

Direct Preference Optimization (DPO) has emerged as a compelling approach for training Large Language Models (LLMs) to adhere to human preferences. However, the performance of DPO…