UNA: A Unified Supervised Framework for Efficient LLM Alignment Across Feedback Types

Zhichao Wang, Bin Bi, Can Huang +7

RL alignment methods, including RLHF and DPO, are primarily based on pairwise preference data. Although scalar or score-based feedback has been collected in some settings, it is ra…