7 papers
AI Alignment From Social Choice Perspectives
Daniel Halpern, Evi Micha, Ariel D. Procaccia +3
Alignment from human feedback uses human judgments about model outputs to steer the behavior of language models after pretraining. When those judgments reflect conflicting views of…
Linear Social Choice with Few Queries: A Moment-Based Approach
Luise Ge, Daniel Halpern, Gregory Kehne +1
Most social choice rules assume access to full rankings, while current alignment practice -- despite aiming for diversity -- typically treats voters as anonymous and comparisons as…
Robust AI Evaluation through Maximal Lotteries
Hadi Khalaf, Serena L. Wang, Daniel Halpern +3
The standard way to evaluate language models on subjective tasks is through pairwise comparisons: an annotator chooses the "better" of two responses to a prompt. Leaderboards aggre…
Pairwise Calibrated Rewards for Pluralistic Alignment
Daniel Halpern, Evi Micha, Ariel D. Procaccia +1
Current alignment pipelines presume a single, universal notion of desirable behavior. However, human preferences often diverge across users, contexts, and cultures. As a result, di…
The Proportional Veto Principle for Approval Ballots
Daniel Halpern, Ariel D. Procaccia, Warut Suksompong
The proportional veto principle, which captures the idea that a candidate vetoed by a large group of voters should not be chosen, has been studied for ranked ballots in single-winn…
Federated Assemblies
Daniel Halpern, Ariel D. Procaccia, Ehud Shapiro +1
A citizens' assembly is a group of people who are randomly selected to represent a larger population in a deliberation. While this approach has successfully strengthened democracy,…
Axioms for AI Alignment from Human Feedback
Luise Ge, Daniel Halpern, Evi Micha +4
In the context of reinforcement learning from human feedback (RLHF), the reward function is generally derived from maximum likelihood estimation of a random utility model based on…