The Potential of the Return Distribution for Exploration in RL
arXiv:1806.04242
Abstract
This paper studies the potential of the return distribution for exploration in deterministic reinforcement learning (RL) environments. We study network losses and propagation mechanisms for Gaussian, Categorical and Gaussian mixture distributions. Combined with exploration policies that leverage this return distribution, we solve, for example, a randomized Chain task of length 100, which has not been reported before when learning with neural networks.
arXiv commentPublished at the Exploration in Reinforcement Learning Workshop at the 35th International Conference on Machine Learning, Stockholm, Sweden
References in corpus (14)
- Deep Exploration via Bootstrapped DQN
- A Distributional Perspective on Reinforcement Learning
- Randomized Prior Functions for Deep Reinforcement Learning
- Efficient Bayes-Adaptive Reinforcement Learning using Sample-Based Search
- Generalization and Exploration via Randomized Value Functions
- Parametric Return Density Estimation for Reinforcement Learning
- A Tutorial on Thompson Sampling
- Learning Multimodal Transition Dynamics for Model-Based Reinforcement Learning
- Bayesian Policy Gradients via Alpha Divergence Dropout Inference
- Efficient exploration with Double Uncertain Value Networks
- Exploration by Distributional Reinforcement Learning
- Assumed Density Filtering Q-learning
- Variational Deep Q Network
- Monte Carlo Tree Search for Asymmetric Trees