A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applications

Wenyi Xiao, Zechuan Wang, Leilei Gan +9

With the rapid advancement of large language models (LLMs), aligning policy models with human preferences has become increasingly critical. Direct Preference Optimization (DPO) has…