Online Meta-Learning

Abstract

A central capability of intelligent systems is the ability to continuously build upon previous experiences to speed up and enhance learning of new tasks. Two distinct research paradigms have studied this question. Meta-learning views this problem as learning a prior over model parameters that is amenable for fast adaptation on a new task, but typically assumes the set of tasks are available together as a batch. In contrast, online (regret based) learning considers a sequential setting in which problems are revealed one after the other, but conventionally train only a single model without any task-specific adaptation. This work introduces an online meta-learning setting, which merges ideas from both the aforementioned paradigms to better capture the spirit and practice of continual lifelong learning. We propose the follow the meta leader algorithm which extends the MAML algorithm to this setting. Theoretically, this work provides an $\mathcal{O}(\log T)$ regret guarantee with only one additional higher order smoothness assumption in comparison to the standard online setting. Our experimental evaluation on three different large-scale tasks suggest that the proposed algorithm significantly outperforms alternatives based on traditional online learning approaches.

ICML 2019. The first two authors contributed equally. Expanded Appendix

Cited by in corpus (57)

• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• Few-Shot Learning via Learning the Representation, Provably
• Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization
• La-MAML: Look-ahead Meta Learning for Continual Learning
• Efficient Continual Learning with Modular Networks and Task-Driven Priors
• Meta-Consolidation for Continual Learning
• Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameters
• Task-Robust Model-Agnostic Meta-Learning
• Differentially Private Meta-Learning
• Meta-Adaptive Nonlinear Control: Theory and Algorithms
• Wandering Within a World: Online Contextualized Few-Shot Learning
• Generalization Bounds For Meta-Learning: An Information-Theoretic Analysis
• Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning
• How Important is the Train-Validation Split in Meta-Learning?
• Meta-Learning with Fewer Tasks through Task Interpolation
• Reset-Free Lifelong Learning with Skill-Space Planning
• StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval
• Weighted Meta-Learning
• Online Structured Meta-learning
• Online Meta-Learning for Multi-Source and Semi-Supervised Domain Adaptation
• Uncertainty Prediction for Deep Sequential Regression Using Meta Models
• Formalizing the Generalization-Forgetting Trade-off in Continual Learning
• On the Power of Multitask Representation Learning in Linear MDP
• Towards mental time travel: a hierarchical memory for reinforcement learning agents
• Embedding Adaptation is Still Needed for Few-Shot Learning
• UFO-BLO: Unbiased First-Order Bilevel Optimization
• Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks
• Learning to Recommend via Meta Parameter Partition
• Few-Shot Unsupervised Continual Learning through Meta-Examples
• Impact of Representation Learning in Linear Bandits
• Provable Representation Learning for Imitation Learning via Bi-level Optimization
• Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets
• Conditional Meta-Learning of Linear Representations
• Bilevel Continual Learning
• The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning
• How Does the Task Landscape Affect MAML Performance?
• Bilevel Optimization for Machine Learning: Algorithm Design and Convergence Analysis
• Streaming Self-Training via Domain-Agnostic Unlabeled Images
• Population-Based Evolution Optimizes a Meta-Learning Objective
• On the Convergence Theory of Debiased Model-Agnostic Meta-Reinforcement Learning
• Incremental Meta-Learning via Indirect Discriminant Alignment
• Online Meta Adaptation for Variable-Rate Learned Image Compression
• No-regret Non-convex Online Meta-Learning
• Teaching to Learn: Sequential Teaching of Agents with Inner States
• Federated Few-Shot Learning with Adversarial Learning
• A Unified Cognitive Learning Framework for Adapting to Dynamic Environment and Tasks
• Continual Backprop: Stochastic Gradient Descent with Persistent Randomness
• Online Continual Adaptation with Active Self-Training
• A Meta-Learning Control Algorithm with Provable Finite-Time Guarantees
• Meta-Learning Guarantees for Online Receding Horizon Learning Control
• Multitask Online Mirror Descent
• Compositional federated learning: Applications in distributionally robust averaging and meta learning
• A method of supervised learning from conflicting data with hidden contexts
• The Sample Complexity of Meta Sparse Regression
• Debiasing a First-order Heuristic for Approximate Bi-level Optimization
• On Data Efficiency of Meta-learning
• Meta Learning in the Continuous Time Limit