Unicorn: Continual Learning with a Universal, Off-policy Agent
arXiv:1802.08294
Abstract
Some real-world domains are best characterized as a single task, but for others this perspective is limiting. Instead, some tasks continually grow in complexity, in tandem with the agent's competence. In continual learning, also referred to as lifelong learning, there are no explicit task boundaries or curricula. As learning agents have become more powerful, continual learning remains one of the frontiers that has resisted quick progress. To test continual learning capabilities we consider a challenging 3D domain with an implicit sequence of tasks and sparse rewards. We propose a novel agent architecture called Unicorn, which demonstrates strong continual learning and outperforms several baseline agents on the proposed domain. The agent achieves this by jointly representing and learning multiple policies efficiently, using a parallel off-policy learning setup.
References in corpus (13)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
- Emergence of Locomotion Behaviours in Rich Environments
- IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
- Hindsight Experience Replay
- Learning by Playing - Solving Sparse Reward Tasks from Scratch
- A Distributional Perspective on Reinforcement Learning
- Grounded Language Learning in a Simulated 3D World
- A Deep Hierarchical Approach to Lifelong Learning in Minecraft
- Reverse Curriculum Generation for Reinforcement Learning
- Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
- The Uncertainty Bellman Equation and Exploration
- Automated Curriculum Learning for Neural Networks
Cited by in corpus (15)
- A survey on intrinsic motivation in reinforcement learning
- Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration
- An empirical investigation of the challenges of real-world reinforcement learning
- CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning
- Robust Reinforcement Learning for Continuous Control with Model Misspecification
- Hindsight policy gradients
- Online Continual Learning on Sequences
- Many-Goals Reinforcement Learning
- Universal Successor Features Approximators
- Multi-task Deep Reinforcement Learning with PopArt
- Deep Reinforcement and InfoMax Learning
- Language Grounding through Social Interactions and Curiosity-Driven Multi-Goal Learning
- Deep Sets for Generalization in RL
- Tokenized Data Markets
- Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes