Expert-augmented actor-critic for ViZDoom and Montezumas Revenge
arXiv:1809.03447
Abstract
We propose an expert-augmented actor-critic algorithm, which we evaluate on two environments with sparse rewards: Montezumas Revenge and a demanding maze from the ViZDoom suite. In the case of Montezumas Revenge, an agent trained with our method achieves very good results consistently scoring above 27,000 points (in many experiments beating the first world). With an appropriate choice of hyperparameters, our algorithm surpasses the performance of the expert data. In a number of experiments, we have observed an unreported bug in Montezumas Revenge which allowed the agent to score more than 800,000 points.
References in corpus (12)
- High-Dimensional Continuous Control Using Generalized Advantage Estimation
- Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
- Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
- Hindsight Experience Replay
- Unifying Count-Based Exploration and Intrinsic Motivation
- Deep Successor Reinforcement Learning
- Observe and Look Further: Achieving Consistent Performance on Atari
- Learning Montezuma's Revenge from a Single Demonstration
- Beating Atari with Natural Language Guided Reinforcement Learning
- Pretraining Deep Actor-Critic Reinforcement Learning Algorithms With Expert Demonstrations
- Regret Minimization for Partially Observable Deep Reinforcement Learning
- TD or not TD: Analyzing the Role of Temporal Differencing in Deep Reinforcement Learning