Adaptive Preference Optimization with Uncertainty-aware Utility Anchor

Xiaobo Wang, Zixia Jia, Jiaqi Li +2

Offline preference optimization methods are efficient for large language models (LLMs) alignment. Direct Preference optimization (DPO)-like learning, one of the most popular approa…