177 citations · 551 across the 52 of their papers we have counts for
32 papers · 1 filter
Transfer Learning in Bandits from Unknown Adaptive Policies: Problem-Dependent Lower and Upper Bounds
Adrien Prevost, Timothee Mathieu, Waris Radji +1
Transfer learning in stochastic multi-armed bandits is typically studied assuming that prior data forms a fixed offline dataset or is generated by a known source. In practice, hist…
Information-Based Exploration via Random Features for Reinforcement Learning
Waris Radji, Odalric-Ambrym Maillard
Representation learning has enabled classical exploration strategies to be extended to deep Reinforcement Learning (RL), but often makes algorithms more complex and theoretical gua…
Bandits for Efficient Experimentation: Adapting to Control Group, Preferences, and Context Drifts
Udvas Das, Waris Radji, Debabrota Basu +1
We consider a variant of the linear contextual stochastic multi-armed bandits, where the learner must provide recommendations to a group of users, each having its personalized pref…
How Hard is it to Confuse a World Model?
Waris Radji, Odalric-Ambrym Maillard
In reinforcement learning (RL) theory, the concept of most confusing instances is central to establishing regret lower bounds, that is, the minimal exploration needed to solve a pr…
The Confusing Instance Principle for Online Linear Quadratic Control
Waris Radji, Odalric-Ambrym Maillard
We revisit the problem of controlling linear systems with quadratic cost under unknown dynamics with model-based reinforcement learning. Traditional methods like Optimism in the Fa…
Asymptotically Optimal Problem-Dependent Bandit Policies for Transfer Learning
Adrien Prevost, Timothee Mathieu, Odalric-Ambrym Maillard
We study the non-contextual multi-armed bandit problem in a transfer learning setting: before any pulls, the learner is given N'_k i.i.d. samples from each source distribution nu'_…
Offline Goal-Conditioned Reinforcement Learning with Projective Quasimetric Planning
Anthony Kobanda, Waris Radji, Mathieu Petitbois +2
Offline Goal-Conditioned Reinforcement Learning seeks to train agents to reach specified goals from previously collected trajectories. Scaling that promises to long-horizon tasks r…
A Continual Offline Reinforcement Learning Benchmark for Navigation Tasks
Anthony Kobanda, Odalric-Ambrym Maillard, Rémy Portelas
Autonomous agents operating in domains such as robotics or video game simulations must adapt to changing tasks without forgetting about the previous ones. This process called Conti…
The regret lower bound for communicating Markov Decision Processes
Victor Boone, Odalric-Ambrym Maillard
This paper is devoted to the extension of the regret lower bound beyond ergodic Markov decision processes (MDPs) in the problem dependent setting. While the regret lower bound for…
Provably Efficient Exploration in Reward Machines with Low Regret
Hippolyte Bourel, Anders Jonsson, Odalric-Ambrym Maillard +2
We study reinforcement learning (RL) for decision processes with non-Markovian reward, in which high-level knowledge of the task in the form of reward machines is available to the…
Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning
Anthony Kobanda, Rémy Portelas, Odalric-Ambrym Maillard +1
We consider a Continual Reinforcement Learning setup, where a learning agent must continuously adapt to new tasks while retaining previously acquired skill sets, with a focus on th…
AdaStop: adaptive statistical testing for sound comparisons of Deep RL agents
Timothée Mathieu, Riccardo Della Vecchia, Alena Shilova +4
Recently, the scientific community has questioned the statistical reproducibility of many empirical results, especially in the field of machine learning. To contribute to the resol…
Bilinear Exponential Family of MDPs: Frequentist Regret Bound with Tractable Exploration and Planning
Reda Ouhamma, Debabrota Basu, Odalric-Ambrym Maillard
We study the problem of episodic reinforcement learning in continuous state-action spaces with unknown rewards and transitions. Specifically, we consider the setting where the rewa…
Collaborative Algorithms for Online Personalized Mean Estimation
Mahsa Asadi, Aurélien Bellet, Odalric-Ambrym Maillard +1
We consider an online estimation problem involving a set of agents. Each agent has access to a (personal) process that generates samples from a real-valued distribution and seeks t…
Bandits Corrupted by Nature: Lower Bounds on Regret and Robust Optimistic Algorithm
Debabrota Basu, Odalric-Ambrym Maillard, Timothée Mathieu
We study the corrupted bandit problem, i.e. a stochastic multi-armed bandit problem with unknown reward distributions, which are heavy-tailed and corrupted by a history-indepen…
Bregman Deviations of Generic Exponential Families
Sayak Ray Chowdhury, Patrick Saux, Odalric-Ambrym Maillard +1
We revisit the method of mixture technique, also known as the Laplace method, to study the concentration phenomenon in generic exponential families. Combining the properties of Bre…
Stochastic Online Linear Regression: the Forward Algorithm to Replace Ridge
Reda Ouhamma, Odalric Maillard, Vianney Perchet
We consider the problem of online linear regression in the stochastic setting. We derive high probability regret bounds for online ridge regression and the forward algorithm. This…
Optimal Thompson Sampling strategies for support-aware CVaR bandits
Dorian Baudry, Romain Gautron, Emilie Kaufmann +1
In this paper we study a multi-arm bandit problem in which the quality of each arm is measured by the Conditional Value at Risk (CVaR) at some level alpha of the reward distributio…
Learning Value Functions in Deep Policy Gradients using Residual Variance
Yannis Flet-Berliac, Reda Ouhamma, Odalric-Ambrym Maillard +1
Policy gradient algorithms have proven to be successful in diverse decision making and control tasks. However, these methods suffer from high sample complexity and instability issu…
Improved Exploration in Factored Average-Reward MDPs
Mohammad Sadegh Talebi, Anders Jonsson, Odalric-Ambrym Maillard
We consider a regret minimization task under the average-reward criterion in an unknown Factored Markov Decision Process (FMDP). More specifically, we consider an FMDP where the st…
Forced-exploration free Strategies for Unimodal Bandits
Hassan Saber, Pierre Ménard, Odalric-Ambrym Maillard
We consider a multi-armed bandit problem specified by a set of Gaussian or Bernoulli distributions endowed with a unimodal structure. Although this problem has been addressed in th…
Tightening Exploration in Upper Confidence Reinforcement Learning
Hippolyte Bourel, Odalric-Ambrym Maillard, Mohammad Sadegh Talebi
The upper confidence reinforcement learning (UCRL2) algorithm introduced in (Jaksch et al., 2010) is a popular method to perform regret minimization in unknown discrete Markov Deci…
Robust-Adaptive Control of Linear Systems: beyond Quadratic Costs
Edouard Leurent, Denis Efimov, Odalric-Ambrym Maillard
We consider the problem of robust and adaptive model predictive control (MPC) of a linear system, with unknown parameters that are learned along the way (adaptive), in a critical s…
Model-Based Reinforcement Learning Exploiting State-Action Equivalence
Mahsa Asadi, Mohammad Sadegh Talebi, Hippolyte Bourel +1
Leveraging an equivalence property in the state-space of a Markov Decision Process (MDP) has been investigated in several studies. This paper studies equivalence structure in the r…