Meta-SGD: Learning to Learn Quickly for Few-Shot Learning

Abstract

Few-shot learning is challenging for learning algorithms that learn each task in isolation and from scratch. In contrast, meta-learning learns from many related tasks a meta-learner that can learn a new task more accurately and faster with fewer examples, where the choice of meta-learners is crucial. In this paper, we develop Meta-SGD, an SGD-like, easily trainable meta-learner that can initialize and adapt any differentiable learner in just one step, on both supervised learning and reinforcement learning. Compared to the popular meta-learner LSTM, Meta-SGD is conceptually simpler, easier to implement, and can be learned more efficiently. Compared to the latest meta-learner MAML, Meta-SGD has a much higher capacity by learning to learn not just the learner initialization, but also the learner update direction and learning rate, all in a single meta-learning process. Meta-SGD shows highly competitive performance for few-shot learning on regression, classification, and reinforcement learning.

reinforcement learning included, 20-way classification on MiniImagenet included

References in corpus (4)

• Theoretical Models of Learning to Learn
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Learning to reinforcement learn
• Learning to Learn: Meta-Critic Networks for Sample Efficient Learning

Cited by in corpus (69)

• SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning
• Finding Task-Relevant Features for Few-Shot Learning by Category Traversal
• Empirical Bayes Transductive Meta-Learning with Synthetic Gradients
• Feature-Critic Networks for Heterogeneous Domain Generalization
• Self-supervised Knowledge Distillation for Few-shot Learning
• Transfer Learning via Contextual Invariants for One-to-Many Cross-Domain Recommendation
• Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning
• Learning from Few Samples: A Survey
• learn2learn: A Library for Meta-Learning Research
• Provable Guarantees for Gradient-Based Meta-Learning
• Personalized Federated Learning using Hypernetworks
• Defining Benchmarks for Continual Few-Shot Learning
• Image Deformation Meta-Networks for One-Shot Learning
• Automated Relational Meta-learning
• Learning Dynamic Alignment via Meta-filter for Few-shot Learning
• Revisiting Meta-Learning as Supervised Learning
• Learning Causal Models Online
• Few-shot Learning for Time-series Forecasting
• Toward Multimodal Model-Agnostic Meta-Learning
• A Generic First-Order Algorithmic Framework for Bi-Level Programming Beyond Lower-Level Singleton
• Region Comparison Network for Interpretable Few-shot Image Classification
• StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval
• Instance Credibility Inference for Few-Shot Learning
• Expert Training: Task Hardness Aware Meta-Learning for Few-Shot Classification
• Meta Reinforcement Learning with Task Embedding and Shared Policy
• Learning from the Past: Continual Meta-Learning via Bayesian Graph Modeling
• LaSO: Label-Set Operations networks for multi-label few-shot learning
• Meta-learning algorithms for Few-Shot Computer Vision
• Regularizing Meta-Learning via Gradient Dropout
• Multi-objective Neural Architecture Search via Non-stationary Policy Gradient
• Few-Shot Knowledge Graph Completion
• Embedding Adaptation is Still Needed for Few-Shot Learning
• Adaptive Task Sampling for Meta-Learning
• Meta-learning with negative learning rates
• Real-Time Object Tracking via Meta-Learning: Efficient Model Adaptation and One-Shot Channel Pruning
• Few-Shot Learning with Intra-Class Knowledge Transfer
• Task-adaptive Neural Process for User Cold-Start Recommendation
• EVO-RL: Evolutionary-Driven Reinforcement Learning
• Intelligence, physics and information -- the tradeoff between accuracy and simplicity in machine learning
• MPLP: Learning a Message Passing Learning Protocol
• ReMP: Rectified Metric Propagation for Few-Shot Learning
• Automatic Validation of Textual Attribute Values in E-commerce Catalog by Learning with Limited Labeled Data
• Contextualizing Enhances Gradient Based Meta Learning
• VIABLE: Fast Adaptation via Backpropagating Learned Loss
• Hierarchical Meta Learning
• Meta-learning for mixed linear regression
• Model-Agnostic Meta-Learning using Runge-Kutta Methods
• Meta-Learning with Network Pruning
• Population-Based Evolution Optimizes a Meta-Learning Objective
• Identifying Physical Law of Hamiltonian Systems via Meta-Learning
• Federated Few-Shot Learning with Adversarial Learning
• Meta-learning One-class Classifiers with Eigenvalue Solvers for Supervised Anomaly Detection
• Personalized Adaptive Meta Learning for Cold-start User Preference Prediction
• MetaAugment: Sample-Aware Data Augmentation Policy Learning
• Variable-Shot Adaptation for Online Meta-Learning
• Multi-level Similarity Learning for Low-Shot Recognition
• Meta Cyclical Annealing Schedule: A Simple Approach to Avoiding Meta-Amortization Error
• Is Fast Adaptation All You Need?
• Learning to Continually Learn Rapidly from Few and Noisy Data
• Few-shot Image Classification with Multi-Facet Prototypes
• On Data Efficiency of Meta-learning
• A Few-Shot Sequential Approach for Object Counting
• One-Shot Image Classification by Learning to Restore Prototypes
• Few-shot Learning for Topic Modeling
• MTL2L: A Context Aware Neural Optimiser
• Beyond Categorical Label Representations for Image Classification
• Cooperative Bi-path Metric for Few-shot Learning
• Class Interference Regularization
• BOML: A Modularized Bilevel Optimization Library in Python for Meta Learning