Selective Preference Optimization via Token-Level Reward Function Estimation

Kailai Yang, Zhiwei Liu, Qianqian Xie +3

Recent advancements in large language model alignment leverage token-level supervisions to perform fine-grained preference optimization. However, existing token-level alignment met…