Exploration in Deep Reinforcement Learning: A Survey

Abstract

This paper reviews exploration techniques in deep reinforcement learning. Exploration techniques are of primary importance when solving sparse reward problems. In sparse reward problems, the reward is rare, which means that the agent will not find the reward often by acting randomly. In such a scenario, it is challenging for reinforcement learning to learn rewards and actions association. Thus more sophisticated exploration methods need to be devised. This review provides a comprehensive overview of existing exploration approaches, which are categorized based on the key contributions as follows reward novel states, reward diverse behaviours, goal-based methods, probabilistic methods, imitation-based methods, safe exploration and random-based methods. Then, the unsolved challenges are discussed to provide valuable future research directions. Finally, the approaches of different categories are compared in terms of complexity, computational effort and overall performance.

References in corpus (8)

• A Brief Survey of Deep Reinforcement Learning
• Parameter Space Noise for Exploration
• Safe Exploration in Continuous Action Spaces
• Trial without Error: Towards Safe Reinforcement Learning via Human Intervention
• Asymmetric self-play for automatic goal discovery in robotic manipulation
• Directed Exploration for Reinforcement Learning
• Learning Abstract Models for Strategic Exploration and Fast Reward Transfer
• Maximum Entropy Diverse Exploration: Disentangling Maximum Entropy Reinforcement Learning