495 citations · 721 across the 24 of their papers we have counts for
19 papers · 1 filter
Learning Dynamics of Continual Learning: A Unified View of Data Attribution, Forgetting, and Plasticity Loss
Yi Ren, Wenlong Deng, Guanzhe Hong +2
Modern language models are likely to be updated throughout their lifetime rather than trained once and frozen. Each update therefore participates in a recurring cycle: decide which…
The Statistical Benefits of Quantile Temporal-Difference Learning for Value Estimation
Mark Rowland, Yunhao Tang, Clare Lyle +3
We study the problem of temporal-difference-based policy evaluation in reinforcement learning. In particular, we analyse the use of a distributional reinforcement learning algorith…
Deep Reinforcement Learning with Plasticity Injection
Evgenii Nikishin, Junhyuk Oh, Georg Ostrovski +4
A growing body of evidence suggests that neural networks employed in deep reinforcement learning (RL) gradually lose their plasticity, the ability to learn from new data; however,…
Understanding plasticity in neural networks
Clare Lyle, Zeyu Zheng, Evgenii Nikishin +3
Plasticity, the ability of a neural network to quickly change its predictions in response to new information, is essential for the adaptability and robustness of deep reinforcement…
Generalization Through the Lens of Learning Dynamics
Clare Lyle
A machine learning (ML) system must learn not only to match the output of a target function on a training set, but also to generalize to novel situations in order to yield accurate…
Understanding Self-Predictive Learning for Reinforcement Learning
Yunhao Tang, Zhaohan Daniel Guo, Pierre Harvey Richemond +13
We study the learning dynamics of self-predictive learning for reinforcement learning, a family of algorithms that learn representations by minimizing the prediction error of their…
Learning Dynamics and Generalization in Reinforcement Learning
Clare Lyle, Mark Rowland, Will Dabney +2
Solving a reinforcement learning (RL) problem poses two competing challenges: fitting a potentially discontinuous value function, and generalizing well to new observations. In this…
Understanding and Preventing Capacity Loss in Reinforcement Learning
Clare Lyle, Mark Rowland, Will Dabney
The reinforcement learning (RL) problem is rife with sources of non-stationarity, making it a notoriously difficult problem domain for the application of neural networks. We identi…
DARTS without a Validation Set: Optimizing the Marginal Likelihood
Miroslav Fil, Binxin Ru, Clare Lyle +1
The success of neural architecture search (NAS) has historically been limited by excessive compute requirements. While modern weight-sharing NAS methods such as DARTS are able to f…
Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning
Jannik Kossen, Neil Band, Clare Lyle +3
We challenge a common assumption underlying most supervised deep learning: that a model makes a prediction depending only on its parameters and the features of a single input. To t…
Robustness to Pruning Predicts Generalization in Deep Neural Networks
Lorenz Kuhn, Clare Lyle, Aidan N. Gomez +2
Existing generalization measures that aim to capture a model's simplicity based on parameter counts or norms fail to explain generalization in overparameterized deep neural network…
On The Effect of Auxiliary Tasks on Representation Dynamics
Clare Lyle, Mark Rowland, Georg Ostrovski +1
While auxiliary tasks play a key role in shaping the representations learnt by reinforcement learning agents, much is still unknown about the mechanisms through which this is achie…
PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning
Angelos Filos, Clare Lyle, Yarin Gal +3
We study reinforcement learning (RL) with no-reward demonstrations, a setting in which an RL agent has access to additional data from the interaction of other agents with the same…
A Bayesian Perspective on Training Speed and Model Selection
Clare Lyle, Lisa Schut, Binxin Ru +2
We take a Bayesian perspective to illustrate a connection between training speed and the marginal likelihood in linear models. This provides two major insights: first, that a measu…
On the Benefits of Invariance in Neural Networks
Clare Lyle, Mark van der Wilk, Marta Kwiatkowska +2
Many real world data analysis problems exhibit invariant structure, and models that take advantage of this structure have shown impressive empirical performance, particularly in de…
Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learning
Andreas Kirsch, Clare Lyle, Yarin Gal
The Information Bottleneck principle offers both a mechanism to explain how deep neural networks train and generalize, as well as a regularized objective with which to train models…
Invariant Causal Prediction for Block MDPs
Amy Zhang, Clare Lyle, Shagun Sodhani +5
Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges. In this paper, we consider the problem o…
A Comparative Analysis of Expected and Distributional Reinforcement Learning
Clare Lyle, Pablo Samuel Castro, Marc G. Bellemare
Since their introduction a year ago, distributional approaches to reinforcement learning (distributional RL) have produced strong results relative to the standard approach which mo…
A Geometric Perspective on Optimal Representations for Reinforcement Learning
Marc G. Bellemare, Will Dabney, Robert Dadashi +6
We propose a new perspective on representation learning in reinforcement learning based on geometric properties of the space of value functions. We leverage this perspective to pro…