Learning to Remember Rare Events

Abstract

Despite recent advances, memory-augmented deep neural networks are still limited when it comes to life-long and one-shot learning, especially in remembering rare events. We present a large-scale life-long memory module for use in deep learning. The module exploits fast nearest-neighbor algorithms for efficiency and thus scales to large memory sizes. Except for the nearest-neighbor query, the module is fully differentiable and trained end-to-end with no extra supervision. It operates in a life-long manner, i.e., without the need to reset it during training. Our memory module can be easily added to any part of a supervised neural network. To show its versatility we add it to a number of networks, from simple convolutional ones tested on image classification to deep sequence-to-sequence and recurrent-convolutional models. In all cases, the enhanced network gains the ability to remember and do life-long one-shot learning. Our module remembers training examples shown many thousands of steps in the past and it can successfully generalize from them. We set new state-of-the-art for one-shot learning on the Omniglot dataset and demonstrate, for the first time, life-long one-shot learning in recurrent neural networks on a large-scale machine translation task.

Conference paper accepted for ICLR'17

References in corpus (2)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Large-scale Simple Question Answering with Memory Networks

Cited by in corpus (14)

• A Two-Stage Approach to Few-Shot Learning for Image Recognition
• Neural Episodic Control
• Few-Shot Learning Through an Information Retrieval Lens
• Meta-Graph: Few Shot Link Prediction via Meta Learning
• Few-shot Learning with Meta Metric Learners
• Few-Shot Video Classification via Temporal Alignment
• Learning to Remember Translation History with a Continuous Cache
• Adaptive Posterior Learning: few-shot learning with a surprise-based memory module
• Expert Training: Task Hardness Aware Meta-Learning for Few-Shot Classification
• On the Role of Dataset Quality and Heterogeneity in Model Confidence
• Making Good on LSTMs' Unfulfilled Promise
• Meta Learning with Differentiable Closed-form Solver for Fast Video Object Segmentation
• Learning to Control Latent Representations for Few-Shot Learning of Named Entities
• Improving Sample Efficiency with Normalized RBF Kernels