Deep Meta-Learning: Learning to Learn in the Concept Space

Abstract

Few-shot learning remains challenging for meta-learning that learns a learning algorithm (meta-learner) from many related tasks. In this work, we argue that this is due to the lack of a good representation for meta-learning, and propose deep meta-learning to integrate the representation power of deep learning into meta-learning. The framework is composed of three modules, a concept generator, a meta-learner, and a concept discriminator, which are learned jointly. The concept generator, e.g. a deep residual net, extracts a representation for each instance that captures its high-level concept, on which the meta-learner performs few-shot learning, and the concept discriminator recognizes the concepts. By learning to learn in the concept space rather than in the complicated instance space, deep meta-learning can substantially improve vanilla meta-learning, which is demonstrated on various few-shot image recognition problems. For example, on 5-way-1-shot image recognition on CIFAR-100 and CUB-200, it improves Matching Nets from 50.53% and 56.53% to 58.18% and 63.47%, improves MAML from 49.28% and 50.45% to 56.65% and 64.63%, and improves Meta-SGD from 53.83% and 53.34% to 61.62% and 66.95%, respectively.

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• An Analysis of Deep Neural Network Models for Practical Applications
• Matching Networks for One Shot Learning
• A Simple Neural Attentive Meta-Learner
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Learning to reinforcement learn
• Learning to Remember Rare Events
• Learning feed-forward one-shot learners

Cited by in corpus (29)

• Federated Meta-Learning with Fast Convergence and Efficient Communication
• Meta-Learning with Latent Embedding Optimization
• Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
• A novel database of Children's Spontaneous Facial Expressions (LIRIS-CSE)
• Meta-Learning with Warped Gradient Descent
• DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning
• Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning
• Learning from Few Samples: A Survey
• Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameters
• Multi-Object Tracking with Siamese Track-RCNN
• Image Deformation Meta-Networks for One-Shot Learning
• Adversarial Feature Hallucination Networks for Few-Shot Learning
• Fast Parametric Learning with Activation Memorization
• A novel machine learning based framework for detection of Autism Spectrum Disorder (ASD)
• LaSO: Label-Set Operations networks for multi-label few-shot learning
• Compositional Few-Shot Recognition with Primitive Discovery and Enhancing
• Continual Local Replacement for Few-shot Learning
• What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data
• Meta Adversarial Perturbations
• Bilevel Optimization for Machine Learning: Algorithm Design and Convergence Analysis
• Revisiting Mid-Level Patterns for Cross-Domain Few-Shot Recognition
• Learning to Detect Few-Shot-Few-Clue Misinformation
• Hierarchical Meta Learning
• Meta-learning for mixed linear regression
• A Hybrid Approach with Optimization and Metric-based Meta-Learner for Few-Shot Learning
• GCCN: Global Context Convolutional Network
• One-Shot Image Classification by Learning to Restore Prototypes
• Augmented Bi-path Network for Few-shot Learning
• Meta-Learning with Adjoint Methods