12 citations · 60 across the 20 of their papers we have counts for
21 papers
A single algorithm for both restless and rested rotting bandits
Julien Seznec, Pierre Ménard, Alessandro Lazaric +1
In many application domains (e.g., recommender systems, intelligent tutoring systems), the rewards associated to the actions tend to decrease over time. This decay is either caused…
Planning in entropy-regularized Markov decision processes and games
Jean-Bastien Grill, Omar Darwiche Domingues, Pierre Ménard +2
We propose SmoothCruiser, a new planning algorithm for estimating the value function in entropy-regularized Markov decision processes and two-player games, given a generative model…
The Harder Path: Last Iterate Convergence for Uncoupled Learning in Zero-Sum Games with Bandit Feedback
Côme Fiegel, Pierre Ménard, Tadashi Kozuno +2
We study the problem of learning in zero-sum matrix games with repeated play and bandit feedback. Specifically, we focus on developing uncoupled algorithms that guarantee, without…
Optimal last-iterate convergence in matrix games with bandit feedback using the log-barrier
Come Fiegel, Pierre Menard, Tadashi Kozuno +2
We study the problem of learning minimax policies in zero-sum matrix games. Fiegel et al. (2025) recently showed that achieving last-iterate convergence in this setting is harder w…
Proximal Point Nash Learning from Human Feedback
Daniil Tiapkin, Daniele Calandriello, Denis Belomestny +5
Traditional Reinforcement Learning from Human Feedback (RLHF) often relies on reward models, frequently assuming preference structures like the Bradley--Terry model, which may not…
Model-free Posterior Sampling via Learning Rate Randomization
Daniil Tiapkin, Denis Belomestny, Daniele Calandriello +6
In this paper, we introduce Randomized Q-learning (RandQL), a novel randomized model-free algorithm for regret minimization in episodic Markov Decision Processes (MDPs). To the bes…
Demonstration-Regularized RL
Daniil Tiapkin, Denis Belomestny, Daniele Calandriello +5
Incorporating expert demonstrations has empirically helped to improve the sample efficiency of reinforcement learning (RL). This paper quantifies theoretically to what extent this…
Sharp Deviations Bounds for Dirichlet Weighted Sums with Application to analysis of Bayesian algorithms
Denis Belomestny, Pierre Menard, Alexey Naumov +2
In this work, we derive sharp non-asymptotic deviation bounds for weighted sums of Dirichlet random variables. These bounds are based on a novel integral representation of the dens…
Fast Rates for Maximum Entropy Exploration
Daniil Tiapkin, Denis Belomestny, Daniele Calandriello +7
We address the challenge of exploration in reinforcement learning (RL) when the agent operates in an unknown environment with sparse or no rewards. In this work, we study the maxim…
Optimistic Posterior Sampling for Reinforcement Learning with Few Samples and Tight Guarantees
Daniil Tiapkin, Denis Belomestny, Daniele Calandriello +6
We consider reinforcement learning in an environment modeled by an episodic, finite, stage-dependent Markov decision process of horizon with states, and actions. The pe…
From Dirichlet to Rubin: Optimistic Exploration in RL without Bonuses
Daniil Tiapkin, Denis Belomestny, Eric Moulines +5
We propose the Bayes-UCBVI algorithm for reinforcement learning in tabular, stage-dependent, episodic Markov decision process: a natural extension of the Bayes-UCB algorithm by Kau…
UCB Momentum Q-learning: Correcting the bias without forgetting
Pierre Menard, Omar Darwiche Domingues, Xuedong Shang +1
We propose UCBMQ, Upper Confidence Bound Momentum Q-learning, a new algorithm for reinforcement learning in tabular and possibly stage-dependent, episodic Markov decision process.…
The Influence of Shape Constraints on the Thresholding Bandit Problem
James Cheshire, Pierre Menard, Alexandra Carpentier
We investigate the stochastic Thresholding Bandit problem (TBP) under several shape constraints. On top of (i) the vanilla, unstructured TBP, we consider the case where (ii) the se…
Kernel-Based Reinforcement Learning: A Finite-Time Analysis
Omar Darwiche Domingues, Pierre Ménard, Matteo Pirotta +2
We consider the exploration-exploitation dilemma in finite-horizon reinforcement learning problems whose state-action space is endowed with a metric. We introduce Kernel-UCBVI, a m…
Non-Asymptotic Pure Exploration by Solving Games
Rémy Degenne, Wouter M. Koolen, Pierre Ménard
Pure exploration (aka active testing) is the fundamental task of sequentially gathering information to answer a query about a stochastic environment. Good algorithms make few mista…
Gradient Ascent for Active Exploration in Bandit Problems
Pierre Ménard
We present a new algorithm based on an gradient ascent for a general Active Exploration bandit problem in the fixed confidence setting. This problem encompasses several well studie…
KL-UCB-switch: optimal regret bounds for stochastic bandits from both a distribution-dependent and a distribution-free viewpoints
Aurélien Garivier, Hédi Hadiji, Pierre Menard +1
We consider -armed stochastic bandits and consider cumulative regret bounds up to time . We are interested in strategies achieving simultaneously a distribution-free regret b…
Thresholding Bandit for Dose-ranging: The Impact of Monotonicity
Aurélien Garivier, Pierre Ménard, Laurent Rossi +1
We analyze the sample complexity of the thresholding bandit problem, with and without the assumption that the mean values of the arms are increasing. In each case, we provide a low…
A minimax and asymptotically optimal algorithm for stochastic bandits
Pierre Ménard, Aurélien Garivier
We propose the kl-UCB ++ algorithm for regret minimization in stochastic bandit models with exponential families of distributions. We prove that it is simultaneously asymptotically…
Fano's inequality for random variables
Sebastien Gerchinovitz, Pierre Ménard, Gilles Stoltz
We extend Fano's inequality, which controls the average probability of events in terms of the average of some --divergences, to work with arbitrary events (not necessarily formi…
Explore First, Exploit Next: The True Shape of Regret in Bandit Problems
Aurélien Garivier, Pierre Ménard, Gilles Stoltz
We revisit lower bounds on the regret in the case of multi-armed bandit problems. We obtain non-asymptotic, distribution-dependent bounds and provide straightforward proofs based o…