187 citations · 456 across the 29 of their papers we have counts for
21 papers · 1 filter
Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weighting
Zhang-Wei Hong, Pulkit Agrawal, Rémi Tachet des Combes +1
Most offline reinforcement learning (RL) algorithms return a target policy maximizing a trade-off between (1) the expected performance gain over the behavior policy that collected…
Behavior Prior Representation learning for Offline Reinforcement Learning
Hongyu Zang, Xin Li, Jie Yu +4
Offline reinforcement learning (RL) struggles in environments with rich and noisy inputs, where the agent only has access to a fixed dataset without environment interactions. Past…
Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learning
Riashat Islam, Hongyu Zang, Anirudh Goyal +6
Goal-conditioned reinforcement learning (RL) is a promising direction for training agents that are capable of solving multiple tasks and reach a diverse set of objectives. How to \…
Incorporating Explicit Uncertainty Estimates into Deep Offline Reinforcement Learning
David Brandfonbrener, Remi Tachet des Combes, Romain Laroche
Most theoretically motivated work in the offline reinforcement learning setting requires precise uncertainty estimates. This requirement restricts the algorithms derived in that wo…
When does return-conditioned supervised learning work for offline reinforcement learning?
David Brandfonbrener, Alberto Bietti, Jacob Buckman +2
Several recent works have proposed a class of algorithms for the offline reinforcement learning (RL) problem that we will refer to as return-conditioned supervised learning (RCSL).…
Non-Markovian policies occupancy measures
Romain Laroche, Remi Tachet des Combes, Jacob Buckman
A central object of study in Reinforcement Learning (RL) is the Markovian policy, in which an agent's actions are chosen from a memoryless probability distribution, conditioned onl…
Beyond the Policy Gradient Theorem for Efficient Policy Updates in Actor-Critic Algorithms
Romain Laroche, Remi Tachet
In Reinforcement Learning, the optimal action at a given state is dependent on policy decisions at subsequent states. As a consequence, the learning targets evolve with time and th…
On the Convergence of SARSA with Linear Function Approximation
Shangtong Zhang, Remi Tachet, Romain Laroche
SARSA, a classical on-policy control algorithm for reinforcement learning, is known to chatter when combined with linear function approximation: SARSA does not diverge but oscillat…
Global Optimality and Finite Sample Analysis of Softmax Off-Policy Actor Critic under State Distribution Mismatch
Shangtong Zhang, Remi Tachet, Romain Laroche
In this paper, we establish the global optimality and convergence rate of an off-policy actor critic algorithm in the tabular setting without using density ratio to correct the dis…
Batched Bandits with Crowd Externalities
Romain Laroche, Othmane Safsafi, Raphael Feraud +1
In Batched Multi-Armed Bandits (BMAB), the policy is not allowed to be updated at each time step. Usually, the setting asserts a maximum number of allowed policy updates and the al…
Dr Jekyll and Mr Hyde: the Strange Case of Off-Policy Policy Updates
Romain Laroche, Remi Tachet
The policy gradient theorem states that the policy should only be updated in states that are visited by the current policy, which leads to insufficient planning in the off-policy s…
Multi-Objective SPIBB: Seldonian Offline Policy Improvement with Safety Constraints in Finite MDPs
Harsh Satija, Philip S. Thomas, Joelle Pineau +1
We study the problem of Safe Policy Improvement (SPI) under constraints in the offline Reinforcement Learning (RL) setting. We consider the scenario where: (i) we have a dataset co…
A Deeper Look at Discounting Mismatch in Actor-Critic Algorithms
Shangtong Zhang, Romain Laroche, Harm van Seijen +2
We investigate the discounting mismatch in actor-critic algorithm implementations from a representation learning perspective. Theoretically, actor-critic algorithms usually have di…
Safe Policy Improvement with an Estimated Baseline Policy
Thiago D. Simão, Romain Laroche, Rémi Tachet des Combes
Previous work has shown the unreliability of existing algorithms in the batch Reinforcement Learning setting, and proposed the theoretically-grounded Safe Policy Improvement with B…
Safe Policy Improvement with Soft Baseline Bootstrapping
Kimia Nadjahi, Romain Laroche, Rémi Tachet des Combes
Batch Reinforcement Learning (Batch RL) consists in training a policy using trajectories collected with another policy, called the behavioural policy. Safe policy improvement (SPI)…
Budgeted Reinforcement Learning in Continuous State Space
Nicolas Carrara, Edouard Leurent, Romain Laroche +3
A Budgeted Markov Decision Process (BMDP) is an extension of a Markov Decision Process to critical applications requiring safety constraints. It relies on a notion of risk implemen…
Decentralized Exploration in Multi-Armed Bandits -- Extended version
Raphaël Féraud, Réda Alami, Romain Laroche
We consider the decentralized exploration problem: a set of players collaborate to identify the best arm by asynchronously interacting with the same stochastic environment. The obj…
Safe Policy Improvement with Baseline Bootstrapping
Romain Laroche, Paul Trichelair, Rémi Tachet des Combes
This paper considers Safe Policy Improvement (SPI) in Batch Reinforcement Learning (Batch RL): from a fixed dataset and without direct access to the true environment, train a polic…
Hybrid Reward Architecture for Reinforcement Learning
Harm van Seijen, Mehdi Fatemi, Joshua Romoff +3
One of the main challenges in reinforcement learning (RL) is generalisation. In typical deep RL methods this is achieved by approximating the optimal value function with a low-dime…
Multi-Advisor Reinforcement Learning
Romain Laroche, Mehdi Fatemi, Joshua Romoff +1
We consider tackling a single-agent RL problem by distributing it to learners. These learners, called advisors, endeavour to solve the problem from a different focus. Their adv…
Separation of Concerns in Reinforcement Learning
Harm van Seijen, Mehdi Fatemi, Joshua Romoff +1
In this paper, we propose a framework for solving a single-agent task by using multiple agents, each focusing on different aspects of the task. This approach has two main advantage…