3 citations · 7 across the 10 of their papers we have counts for
12 papers
What Is Lost in Post-Training? Default Collapse and the Loss of In-Context Steerability Across Diverse Perspectives
Jessica Dierking, Itai Shapira, Niclas Boehmer
AI models serving a heterogeneous population must act on the principles appropriate to each user and context. While post-training has been shown to narrow the views large language…
AI Alignment From Social Choice Perspectives
Daniel Halpern, Evi Micha, Ariel D. Procaccia +3
Alignment from human feedback uses human judgments about model outputs to steer the behavior of language models after pretraining. When those judgments reflect conflicting views of…
Robust AI Evaluation through Maximal Lotteries
Hadi Khalaf, Serena L. Wang, Daniel Halpern +3
The standard way to evaluate language models on subjective tasks is through pairwise comparisons: an annotator chooses the "better" of two responses to a prompt. Leaderboards aggre…
How RLHF Amplifies Sycophancy
Itai Shapira, Gerdus Benade, Ariel D. Procaccia
Large language models often exhibit increased sycophantic behavior after preference-based post-training, showing a stronger tendency to affirm a user's stated or implied belief eve…
Incentives in Federated Learning with Heterogeneous Agents
Ariel D. Procaccia, Han Shao, Itai Shapira
Federated learning promises significant sample-efficiency gains by pooling data across multiple agents, yet incentive misalignment is an obstacle: each update is costly to the cont…
Pairwise Calibrated Rewards for Pluralistic Alignment
Daniel Halpern, Evi Micha, Ariel D. Procaccia +1
Current alignment pipelines presume a single, universal notion of desirable behavior. However, human preferences often diverge across users, contexts, and cultures. As a result, di…
Learning Social Welfare Functions
Kanad Shrikar Pardeshi, Itai Shapira, Ariel D. Procaccia +1
Is it possible to understand or imitate a policy maker's rationale by looking at past decisions they made? We formalize this question as the problem of learning social welfare func…
Bias Detection Via Signaling
Yiling Chen, Tao Lin, Ariel D. Procaccia +2
We introduce and study the problem of detecting whether an agent is updating their prior beliefs given new evidence in an optimal way that is Bayesian, or whether they are biased t…
Axioms for AI Alignment from Human Feedback
Luise Ge, Daniel Halpern, Evi Micha +4
In the context of reinforcement learning from human feedback (RLHF), the reward function is generally derived from maximum likelihood estimation of a random utility model based on…
Generative Social Choice
Sara Fish, Paul Gölz, David C. Parkes +4
The mathematical study of voting, social choice theory, has traditionally only been applicable to choices among a few predetermined alternatives, but not to open-ended decisions su…
Optimal Engagement-Diversity Tradeoffs in Social Media
Fabian Baumann, Daniel Halpern, Ariel D. Procaccia +3
Social media platforms are known to optimize user engagement with the help of algorithms. It is widely understood that this practice gives rise to echo chambers\emdash users are ma…
Expressivity of Shallow and Deep Neural Networks for Polynomial Approximation
Itai Shapira
This study explores the number of neurons required for a Rectified Linear Unit (ReLU) neural network to approximate multivariate monomials. We establish an exponential lower bound…