Finding online neural update rules by learning to remember
arXiv:2003.03124
Abstract
We investigate learning of the online local update rules for neural activations (bodies) and weights (synapses) from scratch. We represent the states of each weight and activation by small vectors, and parameterize their updates using (meta-) neural networks. Different neuron types are represented by different embedding vectors which allows the same two functions to be used for all neurons. Instead of training directly for the objective using evolution or long term back-propagation, as is commonly done in similar systems, we motivate and study a different objective: That of remembering past snippets of experience. We explain how this objective relates to standard back-propagation training and other forms of learning. We train for this objective using short term back-propagation and analyze the performance as a function of both the different network types and the difficulty of the problem. We find that this analysis gives interesting insights onto what constitutes a learning rule. We also discuss how such system could form a natural substrate for addressing topics such as episodic memories, meta-learning and auxiliary objectives.
arXiv comment11 Pages, 1 figure
References in corpus (21)
- Sequence to Sequence Learning with Neural Networks
- Representation Learning with Contrastive Predictive Coding
- Pixel Recurrent Neural Networks
- Dota 2 with Large Scale Deep Reinforcement Learning
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
- On First-Order Meta-Learning Algorithms
- RL: Fast Reinforcement Learning via Slow Reinforcement Learning
- Learning to reinforcement learn
- Large Scale Adversarial Representation Learning
- One-shot Learning with Memory-Augmented Neural Networks
- Meta-Gradient Reinforcement Learning
- Using Fast Weights to Attend to the Recent Past
- Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks
- AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
- From Nodes to Networks: Evolving Recurrent Neural Networks
- Discovery of Useful Questions as Auxiliary Tasks
- Metalearned Neural Memory
- The Kanerva Machine: A Generative Distributed Memory
- Learning to learn with backpropagation of Hebbian plasticity
- Network of Evolvable Neural Units: Evolving to Learn at a Synaptic Level