Expert Training: Task Hardness Aware Meta-Learning for Few-Shot Classification

Abstract

Deep neural networks are highly effective when a large number of labeled samples are available but fail with few-shot classification tasks. Recently, meta-learning methods have received much attention, which train a meta-learner on massive additional tasks to gain the knowledge to instruct the few-shot classification. Usually, the training tasks are randomly sampled and performed indiscriminately, often making the meta-learner stuck into a bad local optimum. Some works in the optimization of deep neural networks have shown that a better arrangement of training data can make the classifier converge faster and perform better. Inspired by this idea, we propose an easy-to-hard expert meta-training strategy to arrange the training tasks properly, where easy tasks are preferred in the first phase, then, hard tasks are emphasized in the second phase. A task hardness aware module is designed and integrated into the training procedure to estimate the hardness of a task based on the distinguishability of its categories. In addition, we explore multiple hardness measurements including the semantic relation, the pairwise Euclidean distance, the Hausdorff distance, and the Hilbert-Schmidt independence criterion. Experimental results on the miniImageNet and tieredImageNetSketch datasets show that the meta-learners can obtain better results with our expert training strategy.

arXiv comment9 pages, 6 figures

References in corpus (13)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Meta-Learning for Semi-Supervised Few-Shot Classification
• Hyperspectral Image Denoising Employing a Spatial-Spectral Deep Residual Convolutional Neural Network
• One-shot Learning with Memory-Augmented Neural Networks
• Learning Robust Global Representations by Penalizing Local Predictive Power
• Learning to Remember Rare Events
• Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examples
• Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning
• Learning to Learn: Meta-Critic Networks for Sample Efficient Learning
• Generative Adversarial Networks for MR-CT Deformable Image Registration

Cited by in corpus (5)

• How Does the Task Landscape Affect MAML Performance?
• PIMNet: A Parallel, Iterative and Mimicking Network for Scene Text Recognition
• Exploring Task Difficulty for Few-Shot Relation Extraction
• Uniform Sampling over Episode Difficulty
• Task Attended Meta-Learning for Few-Shot Learning