MIME: Mutual Information Minimisation Exploration
arXiv:2001.05636
Abstract
We show that reinforcement learning agents that learn by surprise (surprisal) get stuck at abrupt environmental transition boundaries because these transitions are difficult to learn. We propose a counter-intuitive solution that we call Mutual Information Minimising Exploration (MIME) where an agent learns a latent representation of the environment without trying to predict the future states. We show that our agent performs significantly better over sharp transition boundaries while matching the performance of surprisal driven agents elsewhere. In particular, we show state-of-the-art performance on difficult learning games such as Gravitar, Montezuma's Revenge and Doom.
References in corpus (7)
- Playing Atari with Deep Reinforcement Learning
- Trust Region Policy Optimization
- Large-Scale Study of Curiosity-Driven Learning
- Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models
- Exploration by Random Network Distillation
- Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning
- Generative Adversarial Networks are Special Cases of Artificial Curiosity (1990) and also Closely Related to Predictability Minimization (1991)