SeRA: Self-Reviewing and Alignment of Large Language Models using Implicit Reward Margins

Jongwoo Ko, Saket Dingliwal, Bhavana Ganesh +3

Direct alignment algorithms (DAAs), such as direct preference optimization (DPO), have become popular alternatives for Reinforcement Learning from Human Feedback (RLHF) due to thei…