82 citations · 98 across the 26 of their papers we have counts for
29 papers
Learning Rewards in Strategic Games
Antoine Bergerault, Cyrille Kone, Giorgia Ramponi
Understanding the objectives that drive agents' behavior is crucial in strategic environments, where multiple agents interact and adapt to one another. This challenge arises in dom…
Aligning Language Models from User Interactions
Thomas Kleine Buening, Jonas Hübotter, Barna Pásztor +3
Multi-turn user interactions are among the most abundant data produced by language models, yet we lack effective methods to learn from them. While typically discarded, these intera…
Multi-agent imitation learning with function approximation: Linear Markov games and beyond
Luca Viano, Till Freihaut, Emanuele Nevali +3
In this work, we present the first theoretical analysis of multi-agent imitation learning (MAIL) in linear Markov games where both the transition dynamics and each agent's reward f…
MAVRL: Learning Reward Functions from Multiple Feedback Types with Amortized Variational Inference
Raphaël Baur, Yannick Metz, Maria Gkoulta +3
Reward learning typically relies on a single feedback type or combines multiple feedback types using manually weighted loss terms. Currently, it remains unclear how to jointly lear…
Optimal Sample Complexity for Single Time-Scale Actor-Critic with Momentum
Navdeep Kumar, Tehila Dahan, Lior Cohen +4
We establish an optimal sample complexity of for obtaining an -optimal global policy using a single-timescale actor-critic (AC) algorithm in infinite-horizon discoun…
Fine-tuning Behavioral Cloning Policies with Preference-Based Reinforcement Learning
Maël Macuglia, Paul Friedrich, Giorgia Ramponi
Deploying reinforcement learning (RL) in robotics, industry, and health care is blocked by two obstacles: the difficulty of specifying accurate rewards and the risk of unsafe, data…
Rate optimal learning of equilibria from data
Till Freihaut, Luca Viano, Emanuele Nevali +3
We close open theoretical gaps in Multi-Agent Imitation Learning (MAIL) by characterizing the limits of non-interactive MAIL and presenting the first interactive algorithm with nea…
Learning Acrobatic Flight from Preferences
Colin Merk, Ismail Geles, Jiaxu Xing +3
Preference-based reinforcement learning (PbRL) enables agents to learn control policies without requiring manually designed reward functions, making it well-suited for tasks where…
Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Games
David Guzman Piedrahita, Yongjin Yang, Mrinmaya Sachan +3
As large language models (LLMs) are increasingly deployed as autonomous agents, understanding their cooperation and social mechanisms is becoming increasingly important. In particu…
Policy Gradient with Tree Search: Avoiding Local Optimas through Lookahead
Uri Koren, Navdeep Kumar, Uri Gadot +3
Classical policy gradient (PG) methods in reinforcement learning frequently converge to suboptimal local optima, a challenge exacerbated in large or complex environments. This work…
Learning Equilibria from Data: Provably Efficient Multi-Agent Imitation Learning
Till Freihaut, Luca Viano, Volkan Cevher +2
This paper provides the first expert sample complexity characterization for learning a Nash equilibrium from expert data in Markov Games. We show that a new quantity named the sing…
Clustered KL-barycenter design for policy evaluation
Simon Weissmann, Till Freihaut, Claire Vernade +2
In the context of stochastic bandit models, this article examines how to design sample-efficient behavior policies for the importance sampling evaluation of multiple target policie…
Dual Formulation for Non-Rectangular Lp Robust Markov Decision Processes
Navdeep Kumar, Adarsh Gupta, Maxence Mohamed Elfatihi +3
We study robust Markov decision processes (RMDPs) with non-rectangular uncertainty sets, which capture interdependencies across states unlike traditional rectangular models. While…
On Feasible Rewards in Multi-Agent Inverse Reinforcement Learning
Till Freihaut, Giorgia Ramponi
Multi-agent Inverse Reinforcement Learning (MAIRL) aims to recover agent reward functions from expert demonstrations. We characterize the feasible reward set in Markov games, ident…
Learning Collusion in Episodic, Inventory-Constrained Markets
Paul Friedrich, Barna Pásztor, Giorgia Ramponi
Pricing algorithms have demonstrated the capability to learn tacit collusion that is largely unaddressed by current regulations. Their increasing use in markets, including oligopol…
On the Convergence of Single-Timescale Actor-Critic
Navdeep Kumar, Priyank Agrawal, Giorgia Ramponi +2
We analyze the global convergence of the single-timescale actor-critic (AC) algorithm for the infinite-horizon discounted Markov Decision Processes (MDPs) with finite state spaces.…
Preference Elicitation for Offline Reinforcement Learning
Alizée Pace, Bernhard Schölkopf, Gunnar Rätsch +1
Applying reinforcement learning (RL) to real-world problems is often made challenging by the inability to interact with the environment and the difficulty of designing reward funct…
Contextual Bilevel Reinforcement Learning for Incentive Alignment
Vinzenz Thoma, Barna Pasztor, Andreas Krause +2
The optimal policy in various real-world strategic decision-making problems depends both on the environmental configuration and exogenous events. For these settings, we introduce C…
Truly No-Regret Learning in Constrained MDPs
Adrian Müller, Pragnya Alatur, Volkan Cevher +2
Constrained Markov decision processes (CMDPs) are a common way to model safety constraints in reinforcement learning. State-of-the-art methods for efficiently solving CMDPs are bas…
Exploiting Causal Graph Priors with Posterior Sampling for Reinforcement Learning
Mirco Mutti, Riccardo De Santi, Marcello Restelli +2
Posterior sampling allows exploitation of prior knowledge on the environment's transition dynamics to improve the sample efficiency of reinforcement learning. The prior is typicall…
On Imitation in Mean-field Games
Giorgia Ramponi, Pavel Kolev, Olivier Pietquin +3
We explore the problem of imitation learning (IL) in the context of mean-field games (MFGs), where the goal is to imitate the behavior of a population of agents following a Nash eq…
Provably Learning Nash Policies in Constrained Markov Potential Games
Pragnya Alatur, Giorgia Ramponi, Niao He +1
Multi-agent reinforcement learning (MARL) addresses sequential decision-making problems with multiple agents, where each agent optimizes its own objective. In many real-world insta…
Cancellation-Free Regret Bounds for Lagrangian Approaches in Constrained Markov Decision Processes
Adrian Müller, Pragnya Alatur, Giorgia Ramponi +1
Constrained Markov Decision Processes (CMDPs) are one of the common ways to model safe reinforcement learning problems, where constraint functions model the safety objectives. Lagr…
Trust Region Policy Optimization with Optimal Transport Discrepancies: Duality and Algorithm for Continuous Actions
Antonio Terpin, Nicolas Lanzetti, Batuhan Yardim +2
Policy Optimization (PO) algorithms have been proven particularly suited to handle the high-dimensionality of real-world continuous control tasks. In this context, Trust Region Pol…