Learning to learn via Self-Critique

Abstract

In few-shot learning, a machine learning system learns from a small set of labelled examples relating to a specific task, such that it can generalize to new examples of the same task. Given the limited availability of labelled examples in such tasks, we wish to make use of all the information we can. Usually a model learns task-specific information from a small training-set (support-set) to predict on an unlabelled validation set (target-set). The target-set contains additional task-specific information which is not utilized by existing few-shot learning methods. Making use of the target-set examples via transductive learning requires approaches beyond the current methods; at inference time, the target-set contains only unlabelled input data-points, and so discriminative learning cannot be used. In this paper, we propose a framework called Self-Critique and Adapt or SCA, which learns to learn a label-free loss function, parameterized as a neural network. A base-model learns on a support-set using existing methods (e.g. stochastic gradient descent combined with the cross-entropy loss), and then is updated for the incoming target-task using the learnt loss function. This label-free loss function is itself optimized such that the learnt model achieves higher generalization performance. Experiments demonstrate that SCA offers substantially reduced error-rates compared to baselines which only adapt on the support-set, and results in state of the art benchmark performance on Mini-ImageNet and Caltech-UCSD Birds 200.

arXiv commentAccepted in NeurIPS 2019

References in corpus (11)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• A Simple Neural Attentive Meta-Learner
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
• On First-Order Meta-Learning Algorithms
• How Does Batch Normalization Help Optimization?
• Meta-Learning with Latent Embedding Optimization
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
• Learning to Optimize
• Evolved Policy Gradients

Cited by in corpus (7)

• Meta-learning PINN loss functions
• Learning to Impute: A General Framework for Semi-supervised Learning
• Compositional Few-Shot Recognition with Primitive Discovery and Enhancing
• Revisiting Mid-Level Patterns for Cross-Domain Few-Shot Recognition
• Gaussian Process Meta Few-shot Classifier Learning via Linear Discriminant Laplace Approximation
• Novelty-Prepared Few-Shot Classification
• Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning