MPLP: Learning a Message Passing Learning Protocol
arXiv:2007.00970
Abstract
We present a novel method for learning the weights of an artificial neural network - a Message Passing Learning Protocol (MPLP). In MPLP, we abstract every operations occurring in ANNs as independent agents. Each agent is responsible for ingesting incoming multidimensional messages from other agents, updating its internal state, and generating multidimensional messages to be passed on to neighbouring agents. We demonstrate the viability of MPLP as opposed to traditional gradient-based approaches on simple feed-forward neural networks, and present a framework capable of generalizing to non-traditional neural network architectures. MPLP is meta learned using end-to-end gradient-based meta-optimisation. We further discuss the observed properties of MPLP and hypothesize its applicability on various fields of deep learning.
Code at https://github.com/google-research/self-organising-systems/tree/master/mplp; code base link fixed
References in corpus (11)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Neural Message Passing for Quantum Chemistry
- Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
- AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
- Random feedback weights support learning in deep neural networks
- Meta-Learning Update Rules for Unsupervised Representation Learning
- Learning to Continually Learn
- AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
- Low-Shot Learning with Imprinted Weights
- Finding online neural update rules by learning to remember
- Network of Evolvable Neural Units: Evolving to Learn at a Synaptic Level