Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learning
arXiv:2108.10533 · doi:10.3390/s22155845
Abstract
Encouraging exploration is a critical issue in deep reinforcement learning. We investigate the effect of initial entropy that significantly influences the exploration, especially at the earlier stage. Our main observations are as follows: 1) low initial entropy increases the probability of learning failure, and 2) this initial entropy is biased towards a low value that inhibits exploration. Inspired by the investigations, we devise entropy-aware model initialization, a simple yet powerful learning strategy for effective exploration. We show that the devised learning strategy significantly reduces learning failures and enhances performance, stability, and learning speed through experiments.
arXiv comment7 pages, 3 figures, 2 tables
References in corpus (11)
- A Brief Survey of Deep Reinforcement Learning
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Asynchronous Methods for Deep Reinforcement Learning
- Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
- RLlib: Abstractions for Distributed Reinforcement Learning
- Unifying Count-Based Exploration and Intrinsic Motivation
- Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach
- Neural Network Based Reinforcement Learning for Audio-Visual Gaze Control in Human-Robot Interaction
- Maximum Entropy-Regularized Multi-Goal Reinforcement Learning
- Understanding the impact of entropy on policy optimization
- State Entropy Maximization with Random Encoders for Efficient Exploration