Metalearned Neural Memory
arXiv:1907.09720
Abstract
We augment recurrent neural networks with an external memory mechanism that builds upon recent progress in metalearning. We conceptualize this memory as a rapidly adaptable function that we parameterize as a deep neural network. Reading from the neural memory function amounts to pushing an input (the key vector) through the function to produce an output (the value vector). Writing to memory means changing the function; specifically, updating the parameters of the neural network to encode desired information. We leverage training and algorithmic techniques from metalearning to update the neural memory function in one shot. The proposed memory-augmented model achieves strong performance on a variety of learning problems, from supervised question answering to reinforcement learning.
NeurIPS 2019
References in corpus (13)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- A Simple Neural Attentive Meta-Learner
- A Learned Representation For Artistic Style
- Learning to reinforcement learn
- One-Shot Visual Imitation Learning via Meta-Learning
- Direct Feedback Alignment Provides Learning in Deep Neural Networks
- Using Fast Weights to Attend to the Recent Past
- Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes
- Memory Augmented Control Networks
- Learning to Reason with Third-Order Tensor Products
- Metalearning with Hebbian Fast Weights
- Learning Attractor Dynamics for Generative Memory
- Contextual Memory Trees
Cited by in corpus (7)
- Self-Attentive Associative Memory
- Meta-trained agents implement Bayes-optimal agents
- Meta-Learning Deep Energy-Based Memory Models
- Distributed Associative Memory Network with Memory Refreshing Loss
- Finding online neural update rules by learning to remember
- Untangling tradeoffs between recurrence and self-attention in neural networks
- Working Memory Graphs