Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels

Abstract

Recently, different machine learning methods have been introduced to tackle the challenging few-shot learning scenario that is, learning from a small labeled dataset related to a specific task. Common approaches have taken the form of meta-learning: learning to learn on the new problem given the old. Following the recognition that meta-learning is implementing learning in a multi-level model, we present a Bayesian treatment for the meta-learning inner loop through the use of deep kernels. As a result we can learn a kernel that transfers to new tasks; we call this Deep Kernel Transfer (DKT). This approach has many advantages: is straightforward to implement as a single optimizer, provides uncertainty quantification, and does not require estimation of task-specific parameters. We empirically demonstrate that DKT outperforms several state-of-the-art algorithms in few-shot classification, and is the state of the art for cross-domain adaptation and regression. We conclude that complex meta-learning routines can be replaced by a simpler Bayesian model without loss of accuracy.

Advances in Neural Information Processing Systems (NeurIPS 2020, Spotlight)

References in corpus (14)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep Learning in Neural Networks: An Overview
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Improved Regularization of Convolutional Neural Networks with Cutout
• On Calibration of Modern Neural Networks
• A Simple Neural Attentive Meta-Learner
• EMNIST: an extension of MNIST to handwritten letters
• Gaussian Process Kernels for Pattern Discovery and Extrapolation
• Meta-Learning with Implicit Gradients
• Few-Shot Learning with Metric-Agnostic Conditional Embeddings
• Defining Benchmarks for Continual Few-Shot Learning
• Adaptive Deep Kernel Learning
• MetaFun: Meta-Learning with Iterative Functional Updates

Cited by in corpus (4)

• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• Bayesian Active Meta-Learning for Few Pilot Demodulation and Equalization
• Efficient exploration and calibration of a semi-analytical model of galaxy formation with deep learning
• Few-Shot Learning Patterns in Financial Time-Series for Trend-Following Strategies