Diversity-Driven Exploration Strategy for Deep Reinforcement Learning
arXiv:1802.04564
Abstract
Efficient exploration remains a challenging research problem in reinforcement learning, especially when an environment contains large state spaces, deceptive local optima, or sparse rewards. To tackle this problem, we present a diversity-driven approach for exploration, which can be easily combined with both off- and on-policy reinforcement learning algorithms. We show that by simply adding a distance measure to the loss function, the proposed methodology significantly enhances an agent's exploratory behaviors, and thus preventing the policy from being trapped in local optima. We further propose an adaptive scaling method for stabilizing the learning process. Our experimental results in Atari 2600 show that our method outperforms baseline approaches in several tasks in terms of mean scores and exploration efficiency.
Cited by in corpus (17)
- Effective Diversity in Population Based Reinforcement Learning
- Learning-Driven Exploration for Reinforcement Learning
- Diversity Actor-Critic: Sample-Aware Entropy Regularization for Sample-Efficient Exploration
- Attraction-Repulsion Actor-Critic for Continuous Control Reinforcement Learning
- Bayesian Curiosity for Efficient Exploration in Reinforcement Learning
- A Max-Min Entropy Framework for Reinforcement Learning
- A Survey of Exploration Methods in Reinforcement Learning
- DiversityGAN: Diversity-Aware Vehicle Motion Prediction via Latent Semantic Sampling
- Adaptable Agent Populations via a Generative Model of Policies
- Multi-Path Policy Optimization
- Harnessing Distribution Ratio Estimators for Learning Agents with Quality and Diversity
- Hierarchies of Planning and Reinforcement Learning for Robot Navigation
- Effects of Different Optimization Formulations in Evolutionary Reinforcement Learning on Diverse Behavior Generation
- Maximum Entropy Diverse Exploration: Disentangling Maximum Entropy Reinforcement Learning
- A Policy Efficient Reduction Approach to Convex Constrained Deep Reinforcement Learning
- Diverse Exploration via Conjugate Policies for Policy Gradient Methods
- How do Offline Measures for Exploration in Reinforcement Learning behave?