Learning from Few Samples: A Survey

Abstract

Deep neural networks have been able to outperform humans in some cases like image recognition and image classification. However, with the emergence of various novel categories, the ability to continuously widen the learning capability of such networks from limited samples, still remains a challenge. Techniques like Meta-Learning and/or few-shot learning showed promising results, where they can learn or generalize to a novel category/task based on prior knowledge. In this paper, we perform a study of the existing few-shot meta-learning techniques in the computer vision domain based on their method and evaluation metrics. We provide a taxonomy for the techniques and categorize them as data-augmentation, embedding, optimization and semantics based learning for few-shot, one-shot and zero-shot settings. We then describe the seminal work done in each category and discuss their approach towards solving the predicament of learning from few samples. Lastly we provide a comparison of these techniques on the commonly used benchmark datasets: Omniglot, and MiniImagenet, along with a discussion towards the future direction of improving the performance of these techniques towards the final goal of outperforming humans.

17 pages, 10 figures

References in corpus (35)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Sequence to Sequence Learning with Neural Networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Natural Language Processing (almost) from Scratch
• How transferable are features in deep neural networks?
• Neural Discrete Representation Learning
• Tutorial on Variational Autoencoders
• Theoretical Models of Learning to Learn
• Recurrent Models of Visual Attention
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Zero-Shot Learning by Convex Combination of Semantic Embeddings
• Meta-Learning for Semi-Supervised Few-Shot Classification
• End-To-End Memory Networks
• Large-scale Simple Question Answering with Memory Networks
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
• Large-Margin Softmax Loss for Convolutional Neural Networks
• Few-Shot Learning with Graph Neural Networks
• Reinforcement Learning with Deep Energy-Based Policies
• Learning to reinforcement learn
• Billion-scale semi-supervised learning for image classification
• One-shot Learning with Memory-Augmented Neural Networks
• Meta-Learning with Latent Embedding Optimization
• FiLM: Visual Reasoning with a General Conditioning Layer
• Question Answering with Subgraph Embeddings
• Deep Meta-Learning: Learning to Learn in the Concept Space
• Learning to Learn: Meta-Critic Networks for Sample Efficient Learning
• Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
• Improving One-Shot Learning through Fusing Side Information
• Discriminative k-shot learning using probabilistic models
• Human Action Performance using Deep Neuro-Fuzzy Recurrent Attention Model
• Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey
• When Does Self-supervision Improve Few-shot Learning?
• Temporal Difference Variational Auto-Encoder

Cited by in corpus (5)

• A Survey of Human-in-the-loop for Machine Learning
• Trans4E: Link Prediction on Scholarly Knowledge Graphs
• Recommender systems based on graph embedding techniques: A comprehensive review
• Into the Unknown: Active Monitoring of Neural Networks
• Effect of backdoor attacks over the complexity of the latent space distribution