Generalizing from a Few Examples: A Survey on Few-Shot Learning

Abstract

Machine learning has been highly successful in data-intensive applications but is often hampered when the data set is small. Recently, Few-Shot Learning (FSL) is proposed to tackle this problem. Using prior knowledge, FSL can rapidly generalize to new tasks containing only a few samples with supervised information. In this paper, we conduct a thorough survey to fully understand FSL. Starting from a formal definition of FSL, we distinguish FSL from several relevant machine learning problems. We then point out that the core issue in FSL is that the empirical risk minimized is unreliable. Based on how prior knowledge can be used to handle this core issue, we categorize FSL methods from three perspectives: (i) data, which uses prior knowledge to augment the supervised experience; (ii) model, which uses prior knowledge to reduce the size of the hypothesis space; and (iii) algorithm, which uses prior knowledge to alter the search for the best hypothesis in the given hypothesis space. With this taxonomy, we review and discuss the pros and cons of each category. Promising directions, in the aspects of the FSL problem setups, techniques, applications and theories, are also proposed to provide insights for future research.

References in corpus (32)

• Auto-Encoding Variational Bayes
• Overcoming catastrophic forgetting in neural networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Neural Architecture Search with Reinforcement Learning
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• An Overview of Multi-Task Learning in Deep Neural Networks
• Conditional Image Generation with PixelCNN Decoders
• DARTS: Differentiable Architecture Search
• Training Very Deep Networks
• A Simple Neural Attentive Meta-Learner
• Meta-Learning for Semi-Supervised Few-Shot Classification
• AutoAugment: Learning Augmentation Policies from Data
• End-To-End Memory Networks
• A Survey on Multi-Task Learning
• Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
• Delta-encoder: an effective sample synthesis method for few-shot object recognition
• Learning to Remember Rare Events
• Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments
• One-Shot Unsupervised Cross Domain Translation
• Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory
• Attentive Recurrent Comparators
• One-Shot Generalization in Deep Generative Models
• One-Shot Adaptation of Supervised Deep Convolutional Models
• Active One-shot Learning
• Small Sample Learning in Big Data Era
• Learning Algorithms for Active Learning
• Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL
• Improving One-Shot Learning through Fusing Side Information
• A Unified Semantic Embedding: Relating Taxonomies and Attributes
• The effects of negative adaptation in Model-Agnostic Meta-Learning
• Efficient K-Shot Learning with Regularized Deep Networks

Cited by in corpus (65)

• Deep Learning for Anomaly Detection: A Review
• COVID-19 Image Data Collection: Prospective Predictions Are the Future
• Deep Learning Algorithms for Rotating Machinery Intelligent Diagnosis: An Open Source Benchmark Study
• Social physics
• Human Action Recognition from Various Data Modalities: A Review
• A Two-Stage Approach to Few-Shot Learning for Image Recognition
• SMOTified-GAN for class imbalanced pattern classification problems
• Machine learning with neural networks
• Toward Open-World Electroencephalogram Decoding Via Deep Learning: A Comprehensive Survey
• A Transfer Learning Evaluation of Deep Neural Networks for Image Classification
• Trans4E: Link Prediction on Scholarly Knowledge Graphs
• Review of Learning-Assisted Power System Optimization
• Low-Quality Training Data Only? A Robust Framework for Detecting Encrypted Malicious Network Traffic
• Towards Green Automated Machine Learning: Status Quo and Future Directions
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• ProtoSound: A Personalized and Scalable Sound Recognition System for Deaf and Hard-of-Hearing Users
• Machine Learning Techniques for Sensor-based Human Activity Recognition with Data Heterogeneity -- A Review
• Image Classification with Small Datasets: Overview and Benchmark
• Recommender systems based on graph embedding techniques: A comprehensive review
• Improving Generalizability of Graph Anomaly Detection Models via Data Augmentation
• Towards a Robust and Trustworthy Machine Learning System Development: An Engineering Perspective
• Stance Detection: A Practical Guide to Classifying Political Beliefs in Text
• A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques
• Federated Few-Shot Learning for Mobile NLP
• Similarity Learning based Few Shot Learning for ECG Time Series Classification
• Vehicle Attribute Recognition by Appearance: Computer Vision Methods for Vehicle Type, Make and Model Classification
• Effect of Annotation Errors on Drone Detection with YOLOv3
• Heri-Graphs: A Workflow of Creating Datasets for Multi-modal Machine Learning on Graphs of Heritage Values and Attributes with Social Media
• Opportunities and Challenges of Deep Learning Methods for Electrocardiogram Data: A Systematic Review
• Weakly-supervised Compositional FeatureAggregation for Few-shot Recognition
• Probing transfer learning with a model of synthetic correlated datasets
• Channel Phase Processing in Wireless Networks for Human Activity Recognition
• Recent Advances in Named Entity Recognition: A Comprehensive Survey and Comparative Study
• Deep Learning for Insider Threat Detection: Review, Challenges and Opportunities
• Building Trust in Conversational AI: A Comprehensive Review and Solution Architecture for Explainable, Privacy-Aware Systems using LLMs and Knowledge Graph
• MAMO: Memory-Augmented Meta-Optimization for Cold-start Recommendation
• Exathlon: A Benchmark for Explainable Anomaly Detection over Time Series
• ReFine: Re-randomization before Fine-tuning for Cross-domain Few-shot Learning
• Active Image Synthesis for Efficient Labeling
• Deep Graph Similarity Learning: A Survey
• Landing AI on Networks: An equipment vendor viewpoint on Autonomous Driving Networks
• CoLLIE: Continual Learning of Language Grounding from Language-Image Embeddings
• One-Shot GAN Generated Fake Face Detection
• Insightful analysis of historical sources at scales beyond human capabilities using unsupervised Machine Learning and XAI
• Beyond Low Earth Orbit: Biomonitoring, Artificial Intelligence, and Precision Space Health
• Graph-based Interpolation of Feature Vectors for Accurate Few-Shot Classification
• Machine learning with limited data
• MetaFBP: Learning to Learn High-Order Predictor for Personalized Facial Beauty Prediction
• CROP: Towards Distributional-Shift Robust Reinforcement Learning using Compact Reshaped Observation Processing
• Distributed Evolution Strategies Using TPUs for Meta-Learning
• Predicting Missing Information of Key Aspects in Vulnerability Reports
• Faster Optimization-Based Meta-Learning Adaptation Phase
• Semi-Supervised Few-Shot Atomic Action Recognition
• PACIA: Parameter-Efficient Adapter for Few-Shot Molecular Property Prediction
• Fuzzy Simplicial Networks: A Topology-Inspired Model to Improve Task Generalization in Few-shot Learning
• Active Transfer Prototypical Network: An Efficient Labeling Algorithm for Time-Series Data
• When the Few Outweigh the Many: Illicit Content Recognition with Few-Shot Learning
• One Representation to Rule Them All: Identifying Out-of-Support Examples in Few-shot Learning with Generic Representations
• Confusable Learning for Large-class Few-Shot Classification
• Meta-Learning for One-Class Classification with Few Examples using Order-Equivariant Network
• ReMarNet: Conjoint Relation and Margin Learning for Small-Sample Image Classification
• Proxy Network for Few Shot Learning
• Automatic detection of rare pathologies in fundus photographs using few-shot learning
• Studying Evolutionary Solution Adaption Using a Flexibility Benchmark Based on a Metal Cutting Process
• Dynamic Flows on Curved Space Generated by Labeled Data