A Survey of Exploration Methods in Reinforcement Learning
arXiv:2109.00157
Abstract
Exploration is an essential component of reinforcement learning algorithms, where agents need to learn how to predict and control unknown and often stochastic environments. Reinforcement learning agents depend crucially on exploration to obtain informative data for the learning process as the lack of enough information could hinder effective learning. In this article, we provide a survey of modern exploration methods in (Sequential) reinforcement learning, as well as a taxonomy of exploration methods.
References in corpus (43)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Trust Region Policy Optimization
- The information bottleneck method
- Conditional Image Generation with PixelCNN Decoders
- Dueling Network Architectures for Deep Reinforcement Learning
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Analysis of Thompson Sampling for the multi-armed bandit problem
- RL: Fast Reinforcement Learning via Slow Reinforcement Learning
- Reinforcement Learning with Deep Energy-Based Policies
- Massively Parallel Methods for Deep Reinforcement Learning
- Learning to reinforcement learn
- PEGASUS: A Policy Search Method for Large MDPs and POMDPs
- Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models
- Exploration by Random Network Distillation
- Model-Based Bayesian Exploration
- Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
- (More) Efficient Reinforcement Learning via Posterior Sampling
- Memory-based control with recurrent neural networks
- Fast Context Adaptation via Meta-Learning
- Meta-Reinforcement Learning of Structured Exploration Strategies
- Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning
- Deep Exploration via Randomized Value Functions
- Path Integral Policy Improvement with Covariance Matrix Adaptation
- A Bayesian Sampling Approach to Exploration in Reinforcement Learning
- REGAL: A Regularization based Algorithm for Reinforcement Learning in Weakly Communicating MDPs
- A unified view of entropy-regularized Markov decision processes
- Provably Efficient Maximum Entropy Exploration
- Efficient Bayes-Adaptive Reinforcement Learning using Sample-Based Search
- Model-Based Active Exploration
- A Laplacian Framework for Option Discovery in Reinforcement Learning
- Diversity-Driven Exploration Strategy for Deep Reinforcement Learning
- Some Considerations on Learning to Explore via Meta-Reinforcement Learning
- EX2: Exploration with Exemplar Models for Deep Reinforcement Learning
- Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?
- Online Regret Bounds for Undiscounted Continuous Reinforcement Learning
- Better Exploration with Optimistic Actor-Critic
- Successor Uncertainties: Exploration and Uncertainty in Temporal Difference Learning
- Randomized Value Functions via Multiplicative Normalizing Flows
- Optimistic Exploration even with a Pessimistic Initialisation
- Near-optimal Optimistic Reinforcement Learning using Empirical Bernstein Inequalities
- Discovering Options for Exploration by Minimizing Cover Time
- Locally Persistent Exploration in Continuous Control Tasks with Sparse Rewards
- Logistic Q-Learning