Dynamic Few-Shot Visual Learning without Forgetting

Abstract

The human visual system has the remarkably ability to be able to effortlessly learn novel concepts from only a few examples. Mimicking the same behavior on machine learning vision systems is an interesting and very challenging research problem with many practical advantages on real world vision applications. In this context, the goal of our work is to devise a few-shot visual learning system that during test time it will be able to efficiently learn novel categories from only a few training data while at the same time it will not forget the initial categories on which it was trained (here called base categories). To achieve that goal we propose (a) to extend an object recognition system with an attention based few-shot classification weight generator, and (b) to redesign the classifier of a ConvNet model as the cosine similarity function between feature representations and classification weight vectors. The latter, apart from unifying the recognition of both novel and base categories, it also leads to feature representations that generalize better on "unseen" categories. We extensively evaluate our approach on Mini-ImageNet where we manage to improve the prior state-of-the-art on few-shot recognition (i.e., we achieve 56.20% and 73.00% on the 1-shot and 5-shot settings respectively) while at the same time we do not sacrifice any accuracy on the base categories, which is a characteristic that most prior approaches lack. Finally, we apply our approach on the recently introduced few-shot benchmark of Bharath and Girshick [4] where we also achieve state-of-the-art results. The code and models of our paper will be published on: https://github.com/gidariss/FewShotWithoutForgetting

arXiv commentAccepted at CVPR 2018. Code and models will be published on: https://github.com/gidariss/FewShotWithoutForgetting

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Deep Residual Learning for Image Recognition
• Going Deeper with Convolutions
• Matching Networks for One Shot Learning
• A Simple Neural Attentive Meta-Learner
• Low-Shot Learning from Imaginary Data
• Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks
• Low-Shot Learning with Imprinted Weights

Cited by in corpus (27)

• Meta-Learning with Latent Embedding Optimization
• Low-shot Learning via Covariance-Preserving Adversarial Augmentation Networks
• Meta-Learned Confidence for Few-shot Learning
• Meta-DETR: Image-Level Few-Shot Object Detection with Inter-Class Correlation Exploitation
• Multi-Head Self-Attention via Vision Transformer for Zero-Shot Learning
• Reconciling meta-learning and continual learning with online mixtures of tasks
• Variational Prototype Replays for Continual Learning
• L2AE-D: Learning to Aggregate Embeddings for Few-shot Learning with Meta-level Dropout
• Few-Shot Learning as Domain Adaptation: Algorithm and Analysis
• Task-Aware Feature Generation for Zero-Shot Compositional Learning
• Generalized Adaptation for Few-Shot Learning
• Machine learning with limited data
• Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier
• Improving Few-Shot Learning with Auxiliary Self-Supervised Pretext Tasks
• AMP0: Species-Specific Prediction of Anti-microbial Peptides using Zero and Few Shot Learning
• Learning Continually from Low-shot Data Stream
• Generalized Coarse-to-Fine Visual Recognition with Progressive Training
• Defect-GAN: High-Fidelity Defect Synthesis for Automated Defect Inspection
• Representation based and Attention augmented Meta learning
• Coarse-To-Fine Incremental Few-Shot Learning
• On Hard Episodes in Meta-Learning
• Reinforced Attention for Few-Shot Learning and Beyond
• Diversity Transfer Network for Few-Shot Learning
• Composing Text and Image for Image Retrieval - An Empirical Odyssey
• Online Continual Learning in Image Classification: An Empirical Survey
• Offense Detection in Dravidian Languages using Code-Mixing Index based Focal Loss
• Universal-RCNN: Universal Object Detector via Transferable Graph R-CNN