Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace

Abstract

Gradient-based meta-learning methods leverage gradient descent to learn the commonalities among various tasks. While previous such methods have been successful in meta-learning tasks, they resort to simple gradient descent during meta-testing. Our primary contribution is the {\em MT-net}, which enables the meta-learner to learn on each layer's activation space a subspace that the task-specific learner performs gradient descent on. Additionally, a task-specific learner of an {\em MT-net} performs gradient descent with respect to a meta-learned distance metric, which warps the activation space to be more sensitive to task identity. We demonstrate that the dimension of this learned subspace reflects the complexity of the task-specific learner's adaptation task, and also that our model is less sensitive to the choice of initial learning rates than previous gradient-based meta-learning methods. Our method achieves state-of-the-art or comparable performance on few-shot classification and regression tasks.

arXiv commentAccepted to ICML 2018

Cited by in corpus (10)

• Reconciling meta-learning and continual learning with online mixtures of tasks
• L2AE-D: Learning to Aggregate Embeddings for Few-shot Learning with Meta-level Dropout
• Stateless Neural Meta-Learning using Second-Order Gradients
• Few-Shot Learning as Domain Adaptation: Algorithm and Analysis
• Neural Complexity Measures
• Graph Few-shot Learning via Knowledge Transfer
• Discriminative Few-Shot Learning Based on Directional Statistics
• Fast Model Editing at Scale
• Diversity Transfer Network for Few-Shot Learning
• BOML: A Modularized Bilevel Optimization Library in Python for Meta Learning