Less Data, Better Alignment: Data-Centric Multi-Evaluator Agreement for Preference Optimization

Zhengtao Yao, Runhao Li, Xupeng Chen +12

Research on preference optimization often varies the training objective while holding the data fixed. We instead ask whether a small, high-confidence set of on-policy responses can…