3.4k citations · 5k across the 34 of their papers we have counts for
22 papers · 1 filter
Averaging log-likelihoods in direct alignment
Nathan Grinsztajn, Yannis Flet-Berliac, Mohammad Gheshlaghi Azar +8
To better align Large Language Models (LLMs) with human judgment, Reinforcement Learning from Human Feedback (RLHF) learns a reward model and then optimizes it using regularized RL…
Contrastive Policy Gradient: Aligning LLMs on sequence-level scores in a supervised-friendly fashion
Yannis Flet-Berliac, Nathan Grinsztajn, Florian Strub +8
Reinforcement Learning (RL) has been used to finetune Large Language Models (LLMs) using a reward model trained from preference data, to better align with human judgment. The recen…
Self-Improving Robust Preference Optimization
Eugene Choi, Arash Ahmadian, Matthieu Geist +2
Online and offline RLHF methods, such as PPO and DPO, have been highly successful in aligning AI with human preferences. Despite their success, however, these methods suffer from f…
Offline Regularised Reinforcement Learning for Large Language Models Alignment
Pierre Harvey Richemond, Yunhao Tang, Daniel Guo +15
The dominant framework for alignment of large language models (LLM), whether through reinforcement learning from human feedback or direct preference optimisation, is to learn from…
Regularization and Variance-Weighted Regression Achieves Minimax Optimality in Linear MDPs: Theory and Practice
Toshinori Kitamura, Tadashi Kozuno, Yunhao Tang +12
Mirror descent value iteration (MDVI), an abstraction of Kullback-Leibler (KL) and entropy-regularized reinforcement learning (RL), has served as the basis for recent high-performi…
An Analysis of Quantile Temporal-Difference Learning
Mark Rowland, Rémi Munos, Mohammad Gheshlaghi Azar +6
We analyse quantile temporal-difference learning (QTD), a distributional reinforcement learning algorithm that has proven to be a key component in several successful large-scale ap…
Understanding Self-Predictive Learning for Reinforcement Learning
Yunhao Tang, Zhaohan Daniel Guo, Pierre Harvey Richemond +13
We study the learning dynamics of self-predictive learning for reinforcement learning, a family of algorithms that learn representations by minimizing the prediction error of their…
BYOL-Explore: Exploration by Bootstrapped Prediction
Zhaohan Daniel Guo, Shantanu Thakoor, Miruna Pîslar +11
We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually-complex environments. BYOL-Explore learns a world representation, t…
KL-Entropy-Regularized RL with a Generative Model is Minimax Optimal
Tadashi Kozuno, Wenhao Yang, Nino Vieillard +10
In this work, we consider and analyze the sample complexity of model-free reinforcement learning with a generative model. Particularly, we analyze mirror descent value iteration (M…
Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activity
Ran Liu, Mehdi Azabou, Max Dabagia +5
Meaningful and simplified representations of neural activity can yield insights into how and what information is being processed within a neural circuit. However, without labels, f…
Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Prediction
Mehdi Azabou, Mohammad Gheshlaghi Azar, Ran Liu +10
State-of-the-art methods for self-supervised learning (SSL) build representations by maximizing the similarity between different transformed "views" of a sample. Without sufficient…
Large-Scale Representation Learning on Graphs via Bootstrapping
Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar +5
Self-supervised learning provides a promising path towards eliminating the need for costly label information in representation learning on graphs. However, to achieve state-of-the-…
Geometric Entropic Exploration
Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Alaa Saade +7
Exploration is essential for solving complex Reinforcement Learning (RL) tasks. Maximum State-Visitation Entropy (MSVE) formulates the exploration problem as a well-defined policy…
Bootstrap your own latent: A new approach to self-supervised Learning
Jean-Bastien Grill, Florian Strub, Florent Altché +11
We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target…
Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning
Daniel Guo, Bernardo Avila Pires, Bilal Piot +4
Learning a good representation is an essential component for deep reinforcement learning (RL). Representation learning is especially important in multitask and partially observable…
Hindsight Credit Assignment
Anna Harutyunyan, Will Dabney, Thomas Mesnard +8
We consider the problem of efficient credit assignment in reinforcement learning. In order to efficiently and meaningfully utilize new data, we propose to explicitly assign credit…
Meta-learning of Sequential Strategies
Pedro A. Ortega, Jane X. Wang, Mark Rowland +21
In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class. O…
Neural Predictive Belief Representations
Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot +2
Unsupervised representation learning has succeeded with excellent results in many applications. It is an especially powerful tool to learn a good representation of environments wit…
Observe and Look Further: Achieving Consistent Performance on Atari
Tobias Pohlen, Bilal Piot, Todd Hester +10
Despite significant advances in the field of deep Reinforcement Learning (RL), today's algorithms still fail to learn human-level policies consistently over a set of diverse tasks…
Noisy Networks for Exploration
Meire Fortunato, Mohammad Gheshlaghi Azar, Bilal Piot +9
We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to…
On the Sample Complexity of Reinforcement Learning with a Generative Model
Mohammad Gheshlaghi Azar, Remi Munos, Bert Kappen
We consider the problem of learning the optimal action-value function in the discounted-reward Markov decision processes (MDPs). We prove a new PAC bound on the sample-complexity o…
Dynamic Policy Programming
Mohammad Gheshlaghi Azar, Vicenc Gomez, Hilbert J. Kappen
In this paper, we propose a novel policy iteration method, called dynamic policy programming (DPP), to estimate the optimal policy in the infinite-horizon Markov decision processes…