One-shot Learning with Memory-Augmented Neural Networks

Abstract

Despite recent breakthroughs in the applications of deep neural networks, one setting that presents a persistent challenge is that of "one-shot learning." Traditional gradient-based networks require a lot of data to learn, often through extensive iterative training. When new data is encountered, the models must inefficiently relearn their parameters to adequately incorporate the new information without catastrophic interference. Architectures with augmented memory capacities, such as Neural Turing Machines (NTMs), offer the ability to quickly encode and retrieve new information, and hence can potentially obviate the downsides of conventional models. Here, we demonstrate the ability of a memory-augmented neural network to rapidly assimilate new data, and leverage this data to make accurate predictions after only a few samples. We also introduce a new method for accessing an external memory that focuses on memory content, unlike previous methods that additionally use memory location-based focusing mechanisms.

13 pages, 8 figures

Cited by in corpus (100)

• Data Augmentation Generative Adversarial Networks
• Meta-Learning for Semi-Supervised Few-Shot Classification
• Reformer: The Efficient Transformer
• Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
• Learning to Remember Rare Events
• Few-shot classification in Named Entity Recognition Task
• A Self-Attentive model for Knowledge Tracing
• Half a Percent of Labels is Enough: Efficient Animal Detection in UAV Imagery using Deep CNNs and Active Learning
• Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection
• Robust High-dimensional Memory-augmented Neural Networks
• Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning
• Zero-shot Text Classification With Generative Language Models
• Decoupled Novel Object Captioner
• Active One-shot Learning
• Gaussian Prototypical Networks for Few-Shot Learning on Omniglot
• OSLNet: Deep Small-Sample Classification with an Orthogonal Softmax Layer
• Learning Algorithms for Active Learning
• Memory Augmented Neural Networks with Wormhole Connections
• Improving One-Shot Learning through Fusing Side Information
• Comparison of Maximum Likelihood and GAN-based training of Real NVPs
• Generative Temporal Models with Memory
• Learning from Few Samples: A Survey
• Unsupervised Outlier Detection using Memory and Contrastive Learning
• Assume, Augment and Learn: Unsupervised Few-Shot Meta-Learning via Random Labels and Data Augmentation
• Deep Triplet Ranking Networks for One-Shot Recognition
• One-shot Learning for Question-Answering in Gaokao History Challenge
• Incremental Few-Shot Learning with Attention Attractor Networks
• Text normalization using memory augmented neural networks
• Few-Shot Learning with Global Class Representations
• Improved Schemes for Episodic Memory-based Lifelong Learning
• Attentive Neural Processes
• Meta-Learning by the Baldwin Effect
• Concept Learning through Deep Reinforcement Learning with Memory-Augmented Neural Networks
• Penalty Method for Inversion-Free Deep Bilevel Optimization
• Differentiable plasticity: training plastic neural networks with backpropagation
• Generative One-Shot Learning (GOL): A Semi-Parametric Approach to One-Shot Learning in Autonomous Vision
• PixelVAE++: Improved PixelVAE with Discrete Prior
• Task Specific Adversarial Cost Function
• Lung Cancer Screening Using Adaptive Memory-Augmented Recurrent Networks
• Similarity R-C3D for Few-shot Temporal Activity Detection
• Revisiting Few-shot Activity Detection with Class Similarity Control
• Expert Training: Task Hardness Aware Meta-Learning for Few-Shot Classification
• Memorizing Comprehensively to Learn Adaptively: Unsupervised Cross-Domain Person Re-ID with Multi-level Memory
• Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrifices
• KuroNet: Pre-Modern Japanese Kuzushiji Character Recognition with Deep Learning
• Prototypical Clustering Networks for Dermatological Disease Diagnosis
• ToyArchitecture: Unsupervised Learning of Interpretable Models of the World
• Comparison-Based Convolutional Neural Networks for Cervical Cell/Clumps Detection in the Limited Data Scenario
• Meta-Learning Multi-task Communication
• Domain-Adaptive Few-Shot Learning
• Learning to learn with backpropagation of Hebbian plasticity
• Learning from similarity and information extraction from structured documents
• Few-shot Adaptive Faster R-CNN
• Finding online neural update rules by learning to remember
• A Concise Review of Recent Few-shot Meta-learning Methods
• Deep Trustworthy Knowledge Tracing
• Building Machines that Learn and Think for Themselves: Commentary on Lake et al., Behavioral and Brain Sciences, 2017
• On the Importance of Firth Bias Reduction in Few-Shot Classification
• Hierarchical Representation based Query-Specific Prototypical Network for Few-Shot Image Classification
• Rotation Invariance Neural Network
• MatchingGAN: Matching-based Few-shot Image Generation
• Continual Local Replacement for Few-shot Learning
• DAWN: Dual Augmented Memory Network for Unsupervised Video Object Tracking
• Real-Time Visual Object Tracking via Few-Shot Learning
• Few Is Enough: Task-Augmented Active Meta-Learning for Brain Cell Classification
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding
• Visual Tracking via Dynamic Memory Networks
• Understanding and Exploiting Dependent Variables with Deep Metric Learning
• Few-Shot Object Recognition from Machine-Labeled Web Images
• An Empirical Comparison of Deep Learning Models for Knowledge Tracing on Large-Scale Dataset
• Wide and Narrow: Video Prediction from Context and Motion
• A Hybrid Approach with Optimization and Metric-based Meta-Learner for Few-Shot Learning
• Hierarchical Meta Learning
• Continual and Multi-task Reinforcement Learning With Shared Episodic Memory
• Learning to generate classifiers
• AlphaNet: Improving Long-Tail Classification By Combining Classifiers
• Updatable Siamese Tracker with Two-stage One-shot Learning
• Contextual Memory Trees
• Few Shot Learning with Simplex
• Personalizing Pre-trained Models
• Growing a Brain: Fine-Tuning by Increasing Model Capacity
• Retrosynthetic reaction prediction using neural sequence-to-sequence models
• Augmented Memory Networks for Streaming-Based Active One-Shot Learning
• Subjectivity Learning Theory towards Artificial General Intelligence
• One-shot Weakly-Supervised Segmentation in Medical Images
• PICO: Primitive Imitation for COntrol
• Beyond Categorical Label Representations for Image Classification
• Towards Faster k-Nearest-Neighbor Machine Translation
• Efficient Automatic Meta Optimization Search for Few-Shot Learning
• Representation Memorization for Fast Learning New Knowledge without Forgetting
• Latent Cognizance: What Machine Really Learns
• Computational principles of intelligence: learning and reasoning with neural networks
• Video SemNet: Memory-Augmented Video Semantic Network
• Sorted Pooling in Convolutional Networks for One-shot Learning
• Generalized Reinforcement Meta Learning for Few-Shot Optimization
• OLED: One-Class Learned Encoder-Decoder Network with Adversarial Context Masking for Novelty Detection
• Meta-Regularization by Enforcing Mutual-Exclusiveness
• Strategies for Language Identification in Code-Mixed Low Resource Languages
• OR-Net: Pointwise Relational Inference for Data Completion under Partial Observation
• Challenge Closed-book Science Exam: A Meta-learning Based Question Answering System