Robust Multi-Objective Preference Alignment with Online DPO

Raghav Gupta, Ryan Sullivan, Yunxuan Li +2

Multi-objective preference alignment of large language models (LLMs) is critical for developing AI systems that are more configurable, personalizable, helpful, and safe. However, o…