IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL
arXiv:1907.10580 · doi:10.24963/ijcai.2020/280
Abstract
We propose a novel framework to identify sub-goals useful for exploration in sequential decision making tasks under partial observability. We utilize the variational intrinsic control framework (Gregor et.al., 2016) which maximizes empowerment -- the ability to reliably reach a diverse set of states and show how to identify sub-goals as states with high necessary option information through an information theoretic regularizer. Despite being discovered without explicit goal supervision, our sub-goals provide better exploration and sample complexity on challenging grid-world navigation tasks compared to supervised counterparts in prior work.
References in corpus (9)
- Continuous control with deep reinforcement learning
- The information bottleneck method
- Distral: Robust Multitask Reinforcement Learning
- Variational Intrinsic Control
- Variational Option Discovery Algorithms
- A Laplacian Framework for Option Discovery in Reinforcement Learning
- Relevant sparse codes with variational information bottleneck
- InfoBot: Transfer and Exploration via the Information Bottleneck
- Learning to Share and Hide Intentions using Information Regularization