Towards Bridging the Gap Between Offline and Iterative Alignment via Preference Distillation

Wenbo Zhang, Wenzhuo Zhou, Hengrui Cai +1

Direct preference optimization DPO is a promising offline approach for aligning large language models (LLMs) due to its simplicity, computational efficiency, and implicit modeling…