Adaptive Gradient-Based Meta-Learning Methods
arXiv:1906.02717
Abstract
We build a theoretical framework for designing and understanding practical meta-learning methods that integrates sophisticated formalizations of task-similarity with the extensive literature on online convex optimization and sequential prediction algorithms. Our approach enables the task-similarity to be learned adaptively, provides sharper transfer-risk bounds in the setting of statistical learning-to-learn, and leads to straightforward derivations of average-case regret bounds for efficient algorithms in settings where the task-environment changes dynamically or the tasks share a certain geometric structure. We use our theory to modify several popular meta-learning algorithms and improve their meta-test-time performance on standard problems in few-shot learning and federated learning.
NeurIPS 2019
References in corpus (10)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- Theoretical Models of Learning to Learn
- Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
- LEAF: A Benchmark for Federated Settings
- Adaptive Bound Optimization for Online Convex Optimization
- Online Meta-Learning
- Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory
- The Interplay Between Stability and Regret in Online Learning
- Online linear optimization with the log-determinant regularizer