AMoPO: Adaptive Multi-objective Preference Optimization without Reward Models and Reference Models

Qi Liu, Jingqing Ruan, Hao Li +7

Existing multi-objective preference alignment methods for large language models (LLMs) face limitations: (1) the inability to effectively balance various preference dimensions, and…