SPO: Multi-Dimensional Preference Sequential Alignment With Implicit Reward Modeling

Xingzhou Lou, Junge Zhang, Jian Xie +3

Human preference alignment is critical in building powerful and reliable large language models (LLMs). However, current methods either ignore the multi-dimensionality of human pref…