On The Effect of Auxiliary Tasks on Representation Dynamics
arXiv:2102.13089
Abstract
While auxiliary tasks play a key role in shaping the representations learnt by reinforcement learning agents, much is still unknown about the mechanisms through which this is achieved. This work develops our understanding of the relationship between auxiliary tasks, environment structure, and representations by analysing the dynamics of temporal difference algorithms. Through this approach, we establish a connection between the spectral decomposition of the transition operator and the representations induced by a variety of auxiliary tasks. We then leverage insights from these theoretical results to inform the selection of auxiliary tasks for deep reinforcement learning agents in sparse-reward environments.
AISTATS 2021
References in corpus (7)
- Bootstrap your own latent: A new approach to self-supervised Learning
- Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks
- Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learning
- Distributional Reinforcement Learning with Quantile Regression
- Hyperbolic Discounting and Learning over Multiple Horizons
- Representations for Stable Off-Policy Reinforcement Learning
- Unsupervised Basis Function Adaptation for Reinforcement Learning