Metalearning with Hebbian Fast Weights

Abstract

We unify recent neural approaches to one-shot learning with older ideas of associative memory in a model for metalearning. Our model learns jointly to represent data and to bind class labels to representations in a single shot. It builds representations via slow weights, learned across tasks through SGD, while fast weights constructed by a Hebbian learning rule implement one-shot binding for each new task. On the Omniglot, Mini-ImageNet, and Penn Treebank one-shot learning benchmarks, our model achieves state-of-the-art results.

8 pages, 3 figures, 4 tables. arXiv admin note: text overlap with arXiv:1712.09926

References in corpus (4)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• A Simple Neural Attentive Meta-Learner
• Learning Algorithms for Active Learning
• Fast Parametric Learning with Activation Memorization

Cited by in corpus (8)

• One-Shot Instance Segmentation
• Metalearned Neural Memory
• Cross-Domain Few-Shot Learning by Representation Fusion
• Comparison Network for One-Shot Conditional Object Detection
• Reconciling meta-learning and continual learning with online mixtures of tasks
• Testing the Genomic Bottleneck Hypothesis in Hebbian Meta-Learning
• One of these (Few) Things is Not Like the Others
• Representation Memorization for Fast Learning New Knowledge without Forgetting