Online Meta-Learning

Abstract

A central capability of intelligent systems is the ability to continuously build upon previous experiences to speed up and enhance learning of new tasks. Two distinct research paradigms have studied this question. Meta-learning views this problem as learning a prior over model parameters that is amenable for fast adaptation on a new task, but typically assumes the set of tasks are available together as a batch. In contrast, online (regret based) learning considers a sequential setting in which problems are revealed one after the other, but conventionally train only a single model without any task-specific adaptation. This work introduces an online meta-learning setting, which merges ideas from both the aforementioned paradigms to better capture the spirit and practice of continual lifelong learning. We propose the follow the meta leader algorithm which extends the MAML algorithm to this setting. Theoretically, this work provides an $\mathcal{O}(\log T)$ regret guarantee with only one additional higher order smoothness assumption in comparison to the standard online setting. Our experimental evaluation on three different large-scale tasks suggest that the proposed algorithm significantly outperforms alternatives based on traditional online learning approaches.

ICML 2019. The first two authors contributed equally. Expanded Appendix

Cited by in corpus (58)

• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• Few-Shot Learning via Learning the Representation, Provably
• Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization
• La-MAML: Look-ahead Meta Learning for Continual Learning
• Efficient Continual Learning with Modular Networks and Task-Driven Priors
• Meta-Consolidation for Continual Learning
• Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameters
• Task-Robust Model-Agnostic Meta-Learning
• Adaptive Gradient-Based Meta-Learning Methods
• Differentially Private Meta-Learning
• Meta-Adaptive Nonlinear Control: Theory and Algorithms
• Wandering Within a World: Online Contextualized Few-Shot Learning
• Generalization Bounds For Meta-Learning: An Information-Theoretic Analysis
• Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning
• Reset-Free Lifelong Learning with Skill-Space Planning
• Meta-Learning with Fewer Tasks through Task Interpolation
• How Important is the Train-Validation Split in Meta-Learning?
• StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval
• Weighted Meta-Learning
• Online Structured Meta-learning
• Online Meta-Learning for Multi-Source and Semi-Supervised Domain Adaptation
• Uncertainty Prediction for Deep Sequential Regression Using Meta Models
• On the Power of Multitask Representation Learning in Linear MDP
• UFO-BLO: Unbiased First-Order Bilevel Optimization
• Formalizing the Generalization-Forgetting Trade-off in Continual Learning
• Embedding Adaptation is Still Needed for Few-Shot Learning
• Towards mental time travel: a hierarchical memory for reinforcement learning agents
• Few-Shot Unsupervised Continual Learning through Meta-Examples
• Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks
• Provable Representation Learning for Imitation Learning via Bi-level Optimization
• Learning to Recommend via Meta Parameter Partition
• Impact of Representation Learning in Linear Bandits
• How Does the Task Landscape Affect MAML Performance?
• The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning
• Bilevel Continual Learning
• Conditional Meta-Learning of Linear Representations
• Bilevel Optimization for Machine Learning: Algorithm Design and Convergence Analysis
• Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets
• Population-Based Evolution Optimizes a Meta-Learning Objective
• On the Convergence Theory of Debiased Model-Agnostic Meta-Reinforcement Learning
• Streaming Self-Training via Domain-Agnostic Unlabeled Images
• Incremental Meta-Learning via Indirect Discriminant Alignment
• A Unified Cognitive Learning Framework for Adapting to Dynamic Environment and Tasks
• Online Meta Adaptation for Variable-Rate Learned Image Compression
• No-regret Non-convex Online Meta-Learning
• Teaching to Learn: Sequential Teaching of Agents with Inner States
• Federated Few-Shot Learning with Adversarial Learning
• Online Continual Adaptation with Active Self-Training
• Continual Backprop: Stochastic Gradient Descent with Persistent Randomness
• Meta-Learning Guarantees for Online Receding Horizon Learning Control
• Multitask Online Mirror Descent
• A Meta-Learning Control Algorithm with Provable Finite-Time Guarantees
• Compositional federated learning: Applications in distributionally robust averaging and meta learning
• A method of supervised learning from conflicting data with hidden contexts
• Debiasing a First-order Heuristic for Approximate Bi-level Optimization
• On Data Efficiency of Meta-learning
• The Sample Complexity of Meta Sparse Regression
• Meta Learning in the Continuous Time Limit