Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game

Pengyu Cheng, Yifan Yang, Jian Li +5

Human preference alignment is essential to improve the interaction quality of large language models (LLMs). Existing alignment methods depend on manually annotated preference data…