Reconciling meta-learning and continual learning with online mixtures of tasks
arXiv:1812.06080
Abstract
Learning-to-learn or meta-learning leverages data-driven inductive bias to increase the efficiency of learning on a novel task. This approach encounters difficulty when transfer is not advantageous, for instance, when tasks are considerably dissimilar or change over time. We use the connection between gradient-based meta-learning and hierarchical Bayes to propose a Dirichlet process mixture of hierarchical Bayesian models over the parameters of an arbitrary parametric model such as a neural network. In contrast to consolidating inductive biases into a single set of hyperparameters, our approach of task-dependent hyperparameter selection better handles latent distribution shift, as demonstrated on a set of evolving, image-based, few-shot learning benchmarks.
arXiv commentupdated experimental results
References in corpus (18)
- Overcoming catastrophic forgetting in neural networks
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- Continual Learning Through Synaptic Intelligence
- Matching Networks for One Shot Learning
- A Simple Neural Attentive Meta-Learner
- Few-Shot Learning with Graph Neural Networks
- Variational Continual Learning
- Meta-Learning with Latent Embedding Optimization
- TADAM: Task dependent adaptive metric for improved few-shot learning
- Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting
- Bayesian Multitask Learning with Latent Hierarchies
- Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace
- Dynamic Few-Shot Visual Learning without Forgetting
- Few-Shot Learning Through an Information Retrieval Lens
- Discriminative k-shot learning using probabilistic models
- Metalearning with Hebbian Fast Weights
- The effects of negative adaptation in Model-Agnostic Meta-Learning
Cited by in corpus (5)
- Recent Advances of Continual Learning in Computer Vision: An Overview
- Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning
- Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameters
- Non-Gaussian Gaussian Processes for Few-Shot Regression
- How Fine-Tuning Allows for Effective Meta-Learning