Distributionally Robust Listwise Preference Optimization

Xudong Wu, Jian Qian, Pangpang Liu +2

Existing robust preference optimization for language-model alignment mainly studies pairwise supervision and places robustness at the dataset, prompt, or preference-pair level. We…