COMAL: A Convergent Meta-Algorithm for Aligning LLMs with General Preferences

Yixin Liu, Argyris Oikonomou, Weiqiang Zheng +2

Many alignment methods, including reinforcement learning from human feedback (RLHF), rely on the Bradley-Terry reward assumption, which is not always sufficient to capture the full…