2 papers
cs.LG2025
Preference learning made easy: Everything should be understood through win rate
Lily H. Zhang, Rajesh Ranganath
Preference learning, or the task of aligning generative models to preference comparison data, has yet to reach the conceptual maturity of classification, density estimation, etc. T…
cs.LG2024
Preference Learning Algorithms Do Not Learn Preference Rankings
Angelica Chen, Sadhika Malladi, Lily H. Zhang +4
Preference learning algorithms (e.g., RLHF and DPO) are frequently used to steer LLMs to produce generations that are more preferred by humans, but our understanding of their inner…