ComPO: Preference Alignment via Comparison Oracles

Peter Chen, Xi Chen, Wotao Yin +1

Direct alignment methods are increasingly used for aligning large language models (LLMs) with human preferences. However, these methods suffer from the issues of verbosity and like…