Few-Shot Learning via Saliency-guided Hallucination of Samples

Abstract

Learning new concepts from a few of samples is a standard challenge in computer vision. The main directions to improve the learning ability of few-shot training models include (i) a robust similarity learning and (ii) generating or hallucinating additional data from the limited existing samples. In this paper, we follow the latter direction and present a novel data hallucination model. Currently, most datapoint generators contain a specialized network (i.e., GAN) tasked with hallucinating new datapoints, thus requiring large numbers of annotated data for their training in the first place. In this paper, we propose a novel less-costly hallucination method for few-shot learning which utilizes saliency maps. To this end, we employ a saliency network to obtain the foregrounds and backgrounds of available image samples and feed the resulting maps into a two-stream network to hallucinate datapoints directly in the feature space from viable foreground-background combinations. To the best of our knowledge, we are the first to leverage saliency maps for such a task and we demonstrate their usefulness in hallucinating additional datapoints for few-shot learning. Our proposed network achieves the state of the art on publicly available datasets.

arXiv commentIEEE Conference on Computer Vision and Pattern Recognition 2019

References in corpus (4)

• Prototypical Networks for Few-shot Learning
• Low-Shot Learning from Imaginary Data
• Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective
• Feature Generating Networks for Zero-Shot Learning

Cited by in corpus (6)

• Prior Guided Feature Enrichment Network for Few-Shot Segmentation
• Graph-based Interpolation of Feature Vectors for Accurate Few-Shot Classification
• PFENet++: Boosting Few-shot Semantic Segmentation with the Noise-filtered Context-aware Prior Mask
• Weakly-supervised Object Localization for Few-shot Learning and Fine-grained Few-shot Learning
• Machine learning with limited data
• Learning from the Scene and Borrowing from the Rich: Tackling the Long Tail in Scene Graph Generation