156 citations · 595 across the 29 of their papers we have counts for
25 papers · 1 filter
Understanding and Addressing the Pitfalls of Bisimulation-based Representations in Offline Reinforcement Learning
Hongyu Zang, Xin Li, Leiji Zhang +5
While bisimulation-based approaches hold promise for learning robust state representations for Reinforcement Learning (RL) tasks, their efficacy in offline RL tasks has not been up…
Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weighting
Zhang-Wei Hong, Pulkit Agrawal, Rémi Tachet des Combes +1
Most offline reinforcement learning (RL) algorithms return a target policy maximizing a trade-off between (1) the expected performance gain over the behavior policy that collected…
Behavior Prior Representation learning for Offline Reinforcement Learning
Hongyu Zang, Xin Li, Jie Yu +4
Offline reinforcement learning (RL) struggles in environments with rich and noisy inputs, where the agent only has access to a fixed dataset without environment interactions. Past…
Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learning
Riashat Islam, Hongyu Zang, Anirudh Goyal +6
Goal-conditioned reinforcement learning (RL) is a promising direction for training agents that are capable of solving multiple tasks and reach a diverse set of objectives. How to \…
Agent-Controller Representations: Principled Offline RL with Rich Exogenous Information
Riashat Islam, Manan Tomar, Alex Lamb +8
Learning to control an agent from data collected offline in a rich pixel-based visual observation space is vital for real-world applications of reinforcement learning (RL). A major…
Incorporating Explicit Uncertainty Estimates into Deep Offline Reinforcement Learning
David Brandfonbrener, Remi Tachet des Combes, Romain Laroche
Most theoretically motivated work in the offline reinforcement learning setting requires precise uncertainty estimates. This requirement restricts the algorithms derived in that wo…
Non-Markovian policies occupancy measures
Romain Laroche, Remi Tachet des Combes, Jacob Buckman
A central object of study in Reinforcement Learning (RL) is the Markovian policy, in which an agent's actions are chosen from a memoryless probability distribution, conditioned onl…
Beyond the Policy Gradient Theorem for Efficient Policy Updates in Actor-Critic Algorithms
Romain Laroche, Remi Tachet
In Reinforcement Learning, the optimal action at a given state is dependent on policy decisions at subsequent states. As a consequence, the learning targets evolve with time and th…
On the Convergence of SARSA with Linear Function Approximation
Shangtong Zhang, Remi Tachet, Romain Laroche
SARSA, a classical on-policy control algorithm for reinforcement learning, is known to chatter when combined with linear function approximation: SARSA does not diverge but oscillat…
Global Optimality and Finite Sample Analysis of Softmax Off-Policy Actor Critic under State Distribution Mismatch
Shangtong Zhang, Remi Tachet, Romain Laroche
In this paper, we establish the global optimality and convergence rate of an off-policy actor critic algorithm in the tabular setting without using density ratio to correct the dis…
Dr Jekyll and Mr Hyde: the Strange Case of Off-Policy Policy Updates
Romain Laroche, Remi Tachet
The policy gradient theorem states that the policy should only be updated in states that are visited by the current policy, which leads to insufficient planning in the off-policy s…
Decomposed Mutual Information Estimation for Contrastive Representation Learning
Alessandro Sordoni, Nouha Dziri, Hannes Schulz +3
Recent contrastive representation learning methods rely on estimating mutual information (MI) between multiple views of an underlying context. E.g., we can derive multiple views of…
A single gradient step finds adversarial examples on random two-layers neural networks
Sébastien Bubeck, Yeshwanth Cherapanamjeri, Gauthier Gidel +1
Daniely and Schacham recently showed that gradient descent finds adversarial examples on random undercomplete two-layers ReLU neural networks. The term "undercomplete" refers to th…
Adversarial score matching and improved sampling for image generation
Alexia Jolicoeur-Martineau, Rémi Piché-Taillefer, Rémi Tachet des Combes +1
Denoising Score Matching with Annealed Langevin Sampling (DSM-ALS) has recently found success in generative modeling. The approach works by first training a neural network to estim…
A Deeper Look at Discounting Mismatch in Actor-Critic Algorithms
Shangtong Zhang, Romain Laroche, Harm van Seijen +2
We investigate the discounting mismatch in actor-critic algorithm implementations from a representation learning perspective. Theoretically, actor-critic algorithms usually have di…
Deep Reinforcement and InfoMax Learning
Bogdan Mazoure, Remi Tachet des Combes, Thang Doan +2
We begin with the hypothesis that a model-free agent whose representations are predictive of properties of future states (beyond expected rewards) will be more capable of solving a…
Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift
Remi Tachet, Han Zhao, Yu-Xiang Wang +1
Adversarial learning has demonstrated good performance in the unsupervised domain adaptation setting, by learning domain-invariant representations. However, recent work has shown l…
A Reduction from Reinforcement Learning to No-Regret Online Learning
Ching-An Cheng, Remi Tachet des Combes, Byron Boots +1
We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear reg…
Safe Policy Improvement with an Estimated Baseline Policy
Thiago D. Simão, Romain Laroche, Rémi Tachet des Combes
Previous work has shown the unreliability of existing algorithms in the batch Reinforcement Learning setting, and proposed the theoretically-grounded Safe Policy Improvement with B…
Safe Policy Improvement with Soft Baseline Bootstrapping
Kimia Nadjahi, Romain Laroche, Rémi Tachet des Combes
Batch Reinforcement Learning (Batch RL) consists in training a policy using trajectories collected with another policy, called the behavioural policy. Safe policy improvement (SPI)…
On Learning Invariant Representation for Domain Adaptation
Han Zhao, Remi Tachet des Combes, Kun Zhang +1
Due to the ability of deep neural nets to learn rich representations, recent advances in unsupervised domain adaptation have focused on learning domain-invariant features that achi…
An Empirical Study of Example Forgetting during Deep Neural Network Learning
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes +3
Inspired by the phenomenon of catastrophic forgetting, we investigate the learning dynamics of neural networks as they train on single classification tasks. Our goal is to understa…
On the Learning Dynamics of Deep Neural Networks
Remi Tachet, Mohammad Pezeshki, Samira Shabanian +2
While a lot of progress has been made in recent years, the dynamics of learning in deep nonlinear neural networks remain to this day largely misunderstood. In this work, we study t…
Learning Invariances for Policy Generalization
Remi Tachet, Philip Bachman, Harm van Seijen
While recent progress has spawned very powerful machine learning systems, those agents remain extremely specialized and fail to transfer the knowledge they gain to similar yet unse…