3.4k citations · 6.3k across the 41 of their papers we have counts for
24 papers · 1 filter
Learning from negative feedback, or positive feedback or both
Abbas Abdolmaleki, Bilal Piot, Bobak Shahriari +9
Existing preference optimization methods often assume scenarios where paired preference feedback (preferred/positive vs. dis-preferred/negative examples) is available. This require…
Building Math Agents with Multi-Turn Iterative Preference Learning
Wei Xiong, Chengshuai Shi, Jiaming Shen +10
Recent studies have shown that large language models' (LLMs) mathematical problem-solving capabilities can be enhanced by integrating external tools, such as code interpreters, and…
Offline Regularised Reinforcement Learning for Large Language Models Alignment
Pierre Harvey Richemond, Yunhao Tang, Daniel Guo +15
The dominant framework for alignment of large language models (LLM), whether through reinforcement learning from human feedback or direct preference optimisation, is to learn from…
Multi-turn Reinforcement Learning from Preference Human Feedback
Lior Shani, Aviv Rosenberg, Asaf Cassel +10
Reinforcement Learning from Human Feedback (RLHF) has become the standard approach for aligning Large Language Models (LLMs) with human preferences, allowing LLMs to demonstrate re…
Human Alignment of Large Language Models through Online Preference Optimisation
Daniele Calandriello, Daniel Guo, Remi Munos +10
Ensuring alignment of language models' outputs with human preferences is critical to guarantee a useful, safe, and pleasant user experience. Thus, human alignment has been extensiv…
Generalized Preference Optimization: A Unified Approach to Offline Alignment
Yunhao Tang, Zhaohan Daniel Guo, Zeyu Zheng +7
Offline preference optimization allows fine-tuning large models directly from offline data, and has proved effective in recent alignment practices. We propose generalized preferenc…
Unlocking the Power of Representations in Long-term Novelty-based Exploration
Alaa Saade, Steven Kapturowski, Daniele Calandriello +6
We introduce Robust Exploration via Clustering-based Online Density Estimation (RECODE), a non-parametric method for novelty-based exploration that estimates visitation counts for…
The Edge of Orthogonality: A Simple View of What Makes BYOL Tick
Pierre H. Richemond, Allison Tam, Yunhao Tang +3
Self-predictive unsupervised learning methods such as BYOL or SimSiam have shown impressive results, and counter-intuitively, do not collapse to trivial representations. In this wo…
Understanding Self-Predictive Learning for Reinforcement Learning
Yunhao Tang, Zhaohan Daniel Guo, Pierre Harvey Richemond +13
We study the learning dynamics of self-predictive learning for reinforcement learning, a family of algorithms that learn representations by minimizing the prediction error of their…
BYOL-Explore: Exploration by Bootstrapped Prediction
Zhaohan Daniel Guo, Shantanu Thakoor, Miruna Pîslar +11
We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually-complex environments. BYOL-Explore learns a world representation, t…
Shaking the foundations: delusions in sequence models for interaction and control
Pedro A. Ortega, Markus Kunesch, Grégoire Delétang +16
The recent phenomenal success of language models has reinvigorated machine learning research, and large sequence models such as transformers are being applied to a variety of domai…
Geometric Entropic Exploration
Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Alaa Saade +7
Exploration is essential for solving complex Reinforcement Learning (RL) tasks. Maximum State-Visitation Entropy (MSVE) formulates the exploration problem as a well-defined policy…
Bootstrap your own latent: A new approach to self-supervised Learning
Jean-Bastien Grill, Florian Strub, Florent Altché +11
We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target…
Acme: A Research Framework for Distributed Reinforcement Learning
Matthew W. Hoffman, Bobak Shahriari, John Aslanides +36
Deep reinforcement learning (RL) has led to many recent and groundbreaking advances. However, these advances have often come at the cost of both increased scale in the underlying a…
Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning
Daniel Guo, Bernardo Avila Pires, Bilal Piot +4
Learning a good representation is an essential component for deep reinforcement learning (RL). Representation learning is especially important in multitask and partially observable…
Agent57: Outperforming the Atari Human Benchmark
Adrià Puigdomènech Badia, Bilal Piot, Steven Kapturowski +4
Atari games have been a long-standing benchmark in the reinforcement learning (RL) community for the past decade. This benchmark was proposed to test general competency of RL algor…
Never Give Up: Learning Directed Exploration Strategies
Adrià Puigdomènech Badia, Pablo Sprechmann, Alex Vitvitskyi +8
We propose a reinforcement learning agent to solve hard exploration games by learning a range of directed exploratory policies. We construct an episodic memory-based intrinsic rewa…
Hindsight Credit Assignment
Anna Harutyunyan, Will Dabney, Thomas Mesnard +8
We consider the problem of efficient credit assignment in reinforcement learning. In order to efficiently and meaningfully utilize new data, we propose to explicitly assign credit…
Neural Predictive Belief Representations
Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot +2
Unsupervised representation learning has succeeded with excellent results in many applications. It is an especially powerful tool to learn a good representation of environments wit…
Playing the Game of Universal Adversarial Perturbations
Julien Perolat, Mateusz Malinowski, Bilal Piot +1
We study the problem of learning classifiers robust to universal adversarial perturbations. While prior work approaches this problem via robust optimization, adversarial training,…
Observe and Look Further: Achieving Consistent Performance on Atari
Tobias Pohlen, Bilal Piot, Todd Hester +10
Despite significant advances in the field of deep Reinforcement Learning (RL), today's algorithms still fail to learn human-level policies consistently over a set of diverse tasks…
Observational Learning by Reinforcement Learning
Diana Borsa, Bilal Piot, Rémi Munos +1
Observational learning is a type of learning that occurs as a function of observing, retaining and possibly replicating or imitating the behaviour of another agent. It is a core me…
Noisy Networks for Exploration
Meire Fortunato, Mohammad Gheshlaghi Azar, Bilal Piot +9
We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to…
Is the Bellman residual a bad proxy?
Matthieu Geist, Bilal Piot, Olivier Pietquin
This paper aims at theoretically and empirically comparing two standard optimization criteria for Reinforcement Learning: i) maximization of the mean value and ii) minimization of…