Auto-Meta: Automated Gradient Based Meta Learner Search

Abstract

Fully automating machine learning pipelines is one of the key challenges of current artificial intelligence research, since practical machine learning often requires costly and time-consuming human-powered processes such as model design, algorithm development, and hyperparameter tuning. In this paper, we verify that automated architecture search synergizes with the effect of gradient-based meta learning. We adopt the progressive neural architecture search \cite{liu:pnas_google:DBLP:journals/corr/abs-1712-00559} to find optimal architectures for meta-learners. The gradient based meta-learner whose architecture was automatically found achieved state-of-the-art results on the 5-shot 5-way Mini-ImageNet classification problem with $74.65\%$ accuracy, which is $11.54\%$ improvement over the result obtained by the first gradient-based meta-learner called MAML \cite{finn:maml:DBLP:conf/icml/FinnAL17}. To our best knowledge, this work is the first successful neural architecture search implementation in the context of meta learning.

arXiv commentPresented at NIPS 2018 Workshop on Meta-Learning (MetaLearn 2018)

References in corpus (14)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Neural Architecture Search with Reinforcement Learning
• Matching Networks for One Shot Learning
• A Simple Neural Attentive Meta-Learner
• SMASH: One-Shot Model Architecture Search through HyperNetworks
• Learning to reinforcement learn
• Meta-Learning with Latent Embedding Optimization
• Progressive Neural Architecture Search
• DeepArchitect: Automatically Designing and Training Deep Architectures
• Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
• Attentive Recurrent Comparators
• Accelerating Neural Architecture Search using Performance Prediction
• Learning Deep ResNet Blocks Sequentially using Boosting Theory

Cited by in corpus (8)

• AutoML: A Survey of the State-of-the-Art
• NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofing
• Unsupervised Learning via Meta-Learning
• A Comprehensive Overview and Survey of Recent Advances in Meta-Learning
• Provable Guarantees for Gradient-Based Meta-Learning
• Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets
• Rapid Model Architecture Adaption for Meta-Learning
• Is the Meta-Learning Idea Able to Improve the Generalization of Deep Neural Networks on the Standard Supervised Learning?