One Big Net For Everything
arXiv:1802.08864
Abstract
I apply recent work on "learning to think" (2015) and on PowerPlay (2011) to the incremental training of an increasingly general problem solver, continually learning to solve new tasks without forgetting previous skills. The problem solver is a single recurrent neural network (or similar general purpose computer) called ONE. ONE is unusual in the sense that it is trained in various ways, e.g., by black box optimization / reinforcement learning / artificial evolution as well as supervised / unsupervised learning. For example, ONE may learn through neuroevolution to control a robot through environment-changing actions, and learn through unsupervised gradient descent to predict future inputs and vector-valued reward signals as suggested in 1990. User-given tasks can be defined through extra goal-defining input patterns, also proposed in 1990. Suppose ONE has already learned many skills. Now a copy of ONE can be re-trained to learn a new skill, e.g., through neuroevolution without a teacher. Here it may profit from re-using previously learned subroutines, but it may also forget previous skills. Then ONE is retrained in PowerPlay style (2011) on stored input/output traces of (a) ONE's copy executing the new skill and (b) previous instances of ONE whose skills are still considered worth memorizing. Simultaneously, ONE is retrained on old traces (even those of unsuccessful trials) to become a better predictor, without additional expensive interaction with the enviroment. More and more control and prediction skills are thus collapsed into ONE, like in the chunker-automatizer system of the neural history compressor (1991). This forces ONE to relate partially analogous skills (with shared algorithmic information) to each other, creating common subroutines in form of shared subnetworks of ONE, to greatly speed up subsequent learning of additional, novel but algorithmically related skills.
17 pages, 107 references
References in corpus (6)
- Deep Learning in Neural Networks: An Overview
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Show and Tell: A Neural Image Caption Generator
- Multi-column Deep Neural Networks for Image Classification
- Self-Delimiting Neural Networks
- Progressive Reinforcement Learning with Distillation for Multi-Skilled Motion Control
Cited by in corpus (5)
- Will we ever have Conscious Machines?
- Continual Learning with Adaptive Weights (CLAW)
- Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actions
- Self-organization of action hierarchy and compositionality by reinforcement learning with recurrent neural networks
- Generative Adversarial Networks are Special Cases of Artificial Curiosity (1990) and also Closely Related to Predictability Minimization (1991)