Meta Learning Backpropagation And Improving It

Abstract

Many concepts have been proposed for meta learning with neural networks (NNs), e.g., NNs that learn to reprogram fast weights, Hebbian plasticity, learned learning rules, and meta recurrent NNs. Our Variable Shared Meta Learning (VSML) unifies the above and demonstrates that simple weight-sharing and sparsity in an NN is sufficient to express powerful learning algorithms (LAs) in a reusable fashion. A simple implementation of VSML where the weights of a neural network are replaced by tiny LSTMs allows for implementing the backpropagation LA solely by running in forward-mode. It can even meta learn new LAs that differ from online backpropagation and generalize to datasets outside of the meta training distribution without explicit gradient calculation. Introspection reveals that our meta learned LAs learn through fast association in a way that is qualitatively different from gradient descent.

Updated to the NeurIPS 2021 camera ready; fixed typo in eq 4

References in corpus (17)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Learning to reinforcement learn
• Using Fast Weights to Attend to the Recent Past
• AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
• Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples
• Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity
• Discovering Reinforcement Learning Algorithms
• Improving Generalization in Meta Reinforcement Learning using Learned Objectives
• Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity
• One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control
• BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication)
• Meta-learning curiosity algorithms
• Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves
• Finding online neural update rules by learning to remember
• MPLP: Learning a Message Passing Learning Protocol
• Meta-Learning Bidirectional Update Rules

Cited by in corpus (4)

• The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning
• Collective Intelligence for Deep Learning: A Survey of Recent Developments
• Parameter Prediction for Unseen Deep Architectures
• Meta-Learning Bidirectional Update Rules