3 papers
cs.LG2025
Preference learning made easy: Everything should be understood through win rate
Lily H. Zhang, Rajesh Ranganath
Preference learning, or the task of aligning generative models to preference comparison data, has yet to reach the conceptual maturity of classification, density estimation, etc. T…
cs.LG2024
Preference Learning Algorithms Do Not Learn Preference Rankings
Angelica Chen, Sadhika Malladi, Lily H. Zhang +4
Preference learning algorithms (e.g., RLHF and DPO) are frequently used to steer LLMs to produce generations that are more preferred by humans, but our understanding of their inner…
cs.CL2024
Towards Minimal Targeted Updates of Language Models with Targeted Negative Training
Lily H. Zhang, Rajesh Ranganath, Arya Tafvizi
Generative models of language exhibit impressive capabilities but still place non-negligible probability mass over undesirable outputs. In this work, we address the task of updatin…