Bayesian Model-Agnostic Meta-Learning

Abstract

Learning to infer Bayesian posterior from a few-shot dataset is an important step towards robust meta-learning due to the model uncertainty inherent in the problem. In this paper, we propose a novel Bayesian model-agnostic meta-learning method. The proposed method combines scalable gradient-based meta-learning with nonparametric variational inference in a principled probabilistic framework. During fast adaptation, the method is capable of learning complex uncertainty structure beyond a point estimate or a simple Gaussian approximation. In addition, a robust Bayesian meta-update mechanism with a new meta-loss prevents overfitting during meta-update. Remaining an efficient gradient-based meta-learner, the method is also model-agnostic and simple to implement. Experiment results show the accuracy and robustness of the proposed method in various tasks: sinusoidal regression, image classification, active learning, and reinforcement learning.

arXiv commentFirst two authors contributed equally. 15 pages with appendix including experimental details. Accepted in NIPS 2018

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Matching Networks for One Shot Learning
• A Simple Neural Attentive Meta-Learner
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• VIME: Variational Information Maximizing Exploration
• Optimizing Neural Networks with Kronecker-factored Approximate Curvature
• Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
• Bayesian Dark Knowledge
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
• Bayesian Multitask Learning with Latent Hierarchies
• Stein Variational Policy Gradient
• Discriminative k-shot learning using probabilistic models

Cited by in corpus (22)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Diminishing Uncertainty within the Training Pool: Active Learning for Medical Image Segmentation
• Meta-Learning without Memorization
• From Learning to Meta-Learning: Reduced Training Overhead and Complexity for Communication Systems
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• Fed-ensemble: Improving Generalization through Model Ensembling in Federated Learning
• Bayesian Active Meta-Learning for Few Pilot Demodulation and Equalization
• Adaptive Deep Kernel Learning
• The effects of negative adaptation in Model-Agnostic Meta-Learning
• Scalable Bayesian Meta-Learning through Generalized Implicit Gradients
• Compositional ADAM: An Adaptive Compositional Solver
• Data-Efficient Mutual Information Neural Estimator
• Measuring and Harnessing Transference in Multi-Task Learning
• Bayesian Multi-Task Transfer Learning for Soft Prompt Tuning
• Neural Complexity Measures
• Self-organization of action hierarchy and compositionality by reinforcement learning with recurrent neural networks
• Few-Shot Learning Patterns in Financial Time-Series for Trend-Following Strategies
• Information-Theoretic Analysis of Epistemic Uncertainty in Bayesian Meta-learning
• Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayes
• Meta-learning for mixed linear regression
• Transfer Bayesian Meta-learning via Weighted Free Energy Minimization
• Is the Meta-Learning Idea Able to Improve the Generalization of Deep Neural Networks on the Standard Supervised Learning?