Meta-Learning with Implicit Gradients

Abstract

A core capability of intelligent systems is the ability to quickly learn new tasks by drawing on prior experience. Gradient (or optimization) based meta-learning has recently emerged as an effective approach for few-shot learning. In this formulation, meta-parameters are learned in the outer loop, while task-specific models are learned in the inner-loop, by using only a small amount of data from the current task. A key challenge in scaling these approaches is the need to differentiate through the inner loop learning process, which can impose considerable computational and memory burdens. By drawing upon implicit differentiation, we develop the implicit MAML algorithm, which depends only on the solution to the inner level optimization and not the path taken by the inner loop optimizer. This effectively decouples the meta-gradient computation from the choice of inner loop optimizer. As a result, our approach is agnostic to the choice of inner loop optimizer and can gracefully handle many gradient steps without vanishing gradients or memory constraints. Theoretically, we prove that implicit MAML can compute accurate meta-gradients with a memory footprint that is, up to small constant factors, no more than that which is required to compute a single inner loop gradient and at no overall increase in the total computational cost. Experimentally, we show that these benefits of implicit MAML translate into empirical gains on few-shot image recognition benchmarks.

NeurIPS 2019. First two authors contributed equally

Cited by in corpus (118)

• True Few-Shot Learning with Language Models
• TOAN: Target-Oriented Alignment Network for Fine-Grained Image Categorization with Few Labeled Samples
• A novel meta-learning initialization method for physics-informed neural networks
• Revisiting Adversarially Learned Injection Attacks Against Recommender Systems
• Discriminative Adversarial Domain Generalization with Meta-learning based Cross-domain Validation
• Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels
• A Game Theoretic Framework for Model Based Reinforcement Learning
• A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic
• Rectifying the Shortcut Learning of Background for Few-Shot Learning
• Optimizing Millions of Hyperparameters by Implicit Differentiation
• DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning
• Prior-Knowledge and Attention-based Meta-Learning for Few-Shot Learning
• TaskNorm: Rethinking Batch Normalization for Meta-Learning
• Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameters
• Task-Robust Model-Agnostic Meta-Learning
• Cross-Domain Few-Shot Learning by Representation Fusion
• On the Iteration Complexity of Hypergradient Computation
• Penalty Method for Inversion-Free Deep Bilevel Optimization
• Meta-Learning for Few-Shot Land Cover Classification
• On the Outsized Importance of Learning Rates in Local Update Methods
• Online Meta-Critic Learning for Off-Policy Actor-Critic Methods
• Learning to Learn Variational Semantic Memory
• How to Train Your MAML to Excel in Few-Shot Classification
• Learned Initializations for Optimizing Coordinate-Based Neural Representations
• Lower Bounds and Accelerated Algorithms for Bilevel Optimization
• Joint Noise-Tolerant Learning and Meta Camera Shift Adaptation for Unsupervised Person Re-Identification
• Monotone operator equilibrium networks
• On Training Implicit Models
• A Generic First-Order Algorithmic Framework for Bi-Level Programming Beyond Lower-Level Singleton
• A Closer Look at Prototype Classifier for Few-shot Image Classification
• How Important is the Train-Validation Split in Meta-Learning?
• Meta-Learning with Fewer Tasks through Task Interpolation
• Offline Meta-Reinforcement Learning with Advantage Weighting
• Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognition
• Provably Faster Algorithms for Bilevel Optimization
• Modeling and Optimization Trade-off in Meta-learning
• Boosting Few-Shot Learning With Adaptive Margin Loss
• Stateless Neural Meta-Learning using Second-Order Gradients
• Is Support Set Diversity Necessary for Meta-Learning?
• Online Structured Meta-learning
• Multi-Objective Meta Learning
• Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning
• LP-SparseMAP: Differentiable Relaxed Optimization for Sparse Structured Prediction
• Learning to Impute: A General Framework for Semi-supervised Learning
• Modular Meta-Learning with Shrinkage
• Solving Structured Hierarchical Games Using Differential Backward Induction
• Structured Prediction for Conditional Meta-Learning
• Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing
• Online Meta-Learning for Multi-Source and Semi-Supervised Domain Adaptation
• A Sample Complexity Separation between Non-Convex and Convex Meta-Learning
• Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing
• EvoGrad: Efficient Gradient-Based Meta-Learning and Hyperparameter Optimization
• UFO-BLO: Unbiased First-Order Bilevel Optimization
• Meta Learning Black-Box Population-Based Optimizers
• Embedding Adaptation is Still Needed for Few-Shot Learning
• How Fine-Tuning Allows for Effective Meta-Learning
• Convergence Properties of Stochastic Hypergradients
• Meta-Learning for Relative Density-Ratio Estimation
• Learning Graph Structure With A Finite-State Automaton Layer
• Variational Feature Disentangling for Fine-Grained Few-Shot Classification
• Meta-Learning with Neural Tangent Kernels
• Provable Representation Learning for Imitation Learning via Bi-level Optimization
• Few-Shot Unsupervised Continual Learning through Meta-Examples
• Auxiliary Learning by Implicit Differentiation
• Multitask Learning with Single Gradient Step Update for Task Balancing
• Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks
• Randomized Stochastic Variance-Reduced Methods for Multi-Task Stochastic Bilevel Optimization
• Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGD
• A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization
• Graceful Degradation and Related Fields
• Meta Adversarial Perturbations
• Unsupervised Meta-Learning through Latent-Space Interpolation in Generative Models
• Attentive Feature Reuse for Multi Task Meta learning
• ReMP: Rectified Metric Propagation for Few-Shot Learning
• A contrastive rule for meta-learning
• Bi-level Score Matching for Learning Energy-based Latent Variable Models
• How Does the Task Landscape Affect MAML Performance?
• NOVAS: Non-convex Optimization via Adaptive Stochastic Search for End-to-End Learning and Control
• Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets
• The Information Geometry of Unsupervised Reinforcement Learning
• Model-Agnostic Learning to Meta-Learn
• Optimization Induced Equilibrium Networks
• MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures
• Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation
• On the Convergence Theory of Debiased Model-Agnostic Meta-Reinforcement Learning
• Amortized Implicit Differentiation for Stochastic Bilevel Optimization
• Meta-learning for mixed linear regression
• Understanding Deep Architectures with Reasoning Layer
• CMML: Contextual Modulation Meta Learning for Cold-Start Recommendation
• Uniform Priors for Data-Efficient Transfer
• ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning
• Multi-step Estimation for Gradient-based Meta-learning
• Memory Efficient Meta-Learning with Large Images
• GCCN: Global Context Convolutional Network
• Value-Function-based Sequential Minimization for Bi-level Optimization
• Meta-learning Transferable Representations with a Single Target Domain
• Variable-Shot Adaptation for Online Meta-Learning
• Personalized Adaptive Meta Learning for Cold-start User Preference Prediction
• Proximal Mapping for Deep Regularization
• Two Sides of Meta-Learning Evaluation: In vs. Out of Distribution
• Gradients are Not All You Need
• Large-Scale Meta-Learning with Continual Trajectory Shifting
• A Markov Decision Process Approach to Active Meta Learning
• Few-shot Learning for Unsupervised Feature Selection
• BOML: A Modularized Bilevel Optimization Library in Python for Meta Learning
• Task Attended Meta-Learning for Few-Shot Learning
• Uniform Sampling over Episode Difficulty
• Meta-Learning with Adjoint Methods
• A Meta-Learning Control Algorithm with Provable Finite-Time Guarantees
• Gradient-based Hyperparameter Optimization Over Long Horizons
• Camera Distortion-aware 3D Human Pose Estimation in Video with Optimization-based Meta-Learning
• Semi-Supervised Learning with Meta-Gradient
• Covariate Distribution Aware Meta-learning
• Debiasing a First-order Heuristic for Approximate Bi-level Optimization
• Meta-Learning Guarantees for Online Receding Horizon Learning Control
• Eden: A Unified Environment Framework for Booming Reinforcement Learning Algorithms
• Meta-Learning to Improve Pre-Training
• Meta Learning in the Continuous Time Limit