44 citations · 95 across the 36 of their papers we have counts for
39 papers
Inference-Time Nash Alignment
Hadi Hosseini, Debmalya Mandal, Duohan Zhang
Preference-based fine-tuning methods such as RLHF and DPO require substantial compute and large preference datasets. They also need direct access to the model parameters which are…
Local and Global Stability in Performative Reinforcement Learning
Debmalya Mandal
In performative reinforcement learning the deployed policy shapes the environment that generates the learner's future data, and the natural solution concept is a performatively sta…
Meta-Learning Preferences for Multilingual LLM Alignment
Jiaying Lin, Seongho Son, Nam Phuong Tran +3
Unequal availability of human preference data across languages poses a significant challenge for aligning large language models in multilingual settings. To address the lack of suf…
GRAPHLCP: Structure-Aware Localized Conformal Prediction on Graphs
Peyman Baghershahi, Fangxin Wang, Debmalya Mandal +1
Conformal prediction (CP) provides a distribution-free approach to uncertainty quantification with finite-sample guarantees. However, applying CP to graph neural networks (GNNs) re…
Corruption-robust Offline Multi-agent Reinforcement Learning From Human Feedback
Andi Nika, Debmalya Mandal, Parameswaran Kamalaruban +2
We consider robustness against data corruption in offline multi-agent reinforcement learning from human feedback (MARLHF) under a strong-contamination model: given a dataset of…
SP-Rank: A Dataset for Ranked Preferences with Secondary Information
Hadi Hosseini, Debmalya Mandal, Amrit Puhan
We introduce , the first large-scale, publicly available dataset for benchmarking algorithms that leverage both first-order preferences and second-order predictio…
Sparse Offline Reinforcement Learning with Corruption Robustness
Nam Phuong Tran, Andi Nika, Goran Radanovic +2
We investigate robustness to strong data corruption in offline sparse reinforcement learning (RL). In our setting, an adversary may arbitrarily perturb a fraction of the collected…
On Corruption-Robustness in Performative Reinforcement Learning
Vasilis Pollatos, Debmalya Mandal, Goran Radanovic
In performative Reinforcement Learning (RL), an agent faces a policy-dependent environment: the reward and transition functions depend on the agent's policy. Prior work on performa…
Independent Learning in Performative Markov Potential Games
Rilind Sahitaj, Paulius Sasnauskas, Yiğit Yalın +2
Performative Reinforcement Learning (PRL) refers to a scenario in which the deployed policy changes the reward and transition dynamics of the underlying environment. In this work,…
Policy Teaching via Data Poisoning in Learning from Human Preferences
Andi Nika, Jonathan Nöther, Debmalya Mandal +3
We study data poisoning attacks in learning from human preferences. More specifically, we consider the problem of teaching/enforcing a target policy by synthesizing pre…
Strategyproof Reinforcement Learning from Human Feedback
Thomas Kleine Buening, Jiarui Gan, Debmalya Mandal +1
We study Reinforcement Learning from Human Feedback (RLHF) in settings where multiple labelers may strategically misreport feedback to steer the learned policy toward their own pre…
Distributionally Robust Reinforcement Learning with Human Feedback
Debmalya Mandal, Paulius Sasnauskas, Goran Radanovic
Reinforcement learning from human feedback (RLHF) has evolved to be one of the main methods for fine-tuning large language models (LLMs). However, existing RLHF methods are non-rob…
Surprisingly Popular Voting for Concentric Rank-Order Models
Hadi Hosseini, Debmalya Mandal, Amrit Puhan
An important problem on social information sites is the recovery of ground truth from individual reports when the experts are in the minority. The wisdom of the crowd, i.e. the col…
Performative Reinforcement Learning with Linear Markov Decision Process
Debmalya Mandal, Goran Radanovic
We study the setting of \emph{performative reinforcement learning} where the deployed policy affects both the reward, and the transition of the underlying Markov decision process.…
The Surprising Effectiveness of SP Voting with Partial Preferences
Hadi Hosseini, Debmalya Mandal, Amrit Puhan
We consider the problem of recovering the ground truth ordering (ranking, top-, or others) over a large number of alternatives. The wisdom of crowd is a heuristic approach based…
Symmetric Linear Bandits with Hidden Symmetry
Nam Phuong Tran, The Anh Ta, Debmalya Mandal +1
High-dimensional linear bandits with low-dimensional structure have received considerable attention in recent studies due to their practical significance. The most common structure…
Corruption-Robust Offline Two-Player Zero-Sum Markov Games
Andi Nika, Debmalya Mandal, Adish Singla +1
We study data corruption robustness in offline two-player zero-sum Markov games. Given a dataset of realized trajectories of two players, an adversary is allowed to modify an -f…
Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferences
Andi Nika, Debmalya Mandal, Parameswaran Kamalaruban +3
In this paper, we take a step towards a deeper understanding of learning from human preferences by systematically comparing the paradigm of reinforcement learning from human feedba…
Performative Reinforcement Learning in Gradually Shifting Environments
Ben Rank, Stelios Triantafyllou, Debmalya Mandal +1
When Reinforcement Learning (RL) agents are deployed in practice, they might impact their environment and change its dynamics. We propose a new framework to model this phenomenon,…
Learning the Expected Core of Strictly Convex Stochastic Cooperative Games
Nam Phuong Tran, The Anh Ta, Shuqing Shi +3
Reward allocation, also known as the credit assignment problem, has been an important topic in economics, engineering, and machine learning. An important concept in reward allocati…
Corruption Robust Offline Reinforcement Learning with Human Feedback
Debmalya Mandal, Andi Nika, Parameswaran Kamalaruban +2
We study data corruption robustness for reinforcement learning with human feedback (RLHF) in an offline setting. Given an offline dataset of pairs of trajectories along with feedba…
Agent-Specific Effects: A Causal Effect Propagation Analysis in Multi-Agent MDPs
Stelios Triantafyllou, Aleksa Sukovic, Debmalya Mandal +1
Establishing causal relationships between actions and outcomes is fundamental for accountable multi-agent decision-making. However, interpreting and quantifying agents' contributio…
Markov Decision Processes with Time-Varying Geometric Discounting
Jiarui Gan, Annika Hennes, Rupak Majumdar +2
Canonical models of Markov decision processes (MDPs) usually consider geometric discounting based on a constant discount factor. While this standard modeling approach has led to ma…
Stochastic Principal-Agent Problems: Efficient Computation and Learning
Jiarui Gan, Rupak Majumdar, Debmalya Mandal +1
We introduce a stochastic principal-agent model. A principal and an agent interact in a stochastic environment, each privy to observations about the state not available to the othe…