Adaptive Deep Kernel Learning

Abstract

Deep kernel learning provides an elegant and principled framework for combining the structural properties of deep learning algorithms with the flexibility of kernel methods. By means of a deep neural network, we learn a parametrized kernel operator that can be combined with a differentiable kernel algorithm during inference. While previous work within this framework has focused on learning a single kernel for large datasets, we learn a kernel family for a variety of few-shot regression tasks. Compared to single deep kernel learning, our algorithm enables the identification of the appropriate kernel for each task during inference. As such, it is well adapted for complex task distributions in a few-shot learning setting, which we demonstrate by comparing against existing state-of-the-art algorithms using real-world, few-shot regression tasks related to the field of drug discovery.

References in corpus (13)

• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• MINE: Mutual Information Neural Estimation
• Few-Shot Learning with Graph Neural Networks
• Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)
• A Closer Look at Few-shot Classification
• Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination
• Bayesian Model-Agnostic Meta-Learning
• Conditional Neural Processes
• Attentive Neural Processes

Cited by in corpus (10)

• Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• Bayesian Few-Shot Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes
• Information Theoretic Meta Learning with Gaussian Processes
• Non-Gaussian Gaussian Processes for Few-Shot Regression
• Few-shot Learning for Spatial Regression
• Fast Adaptation with Linearized Neural Networks
• End-to-End Learning of Deep Kernel Acquisition Functions for Bayesian Optimization
• Local Nonparametric Meta-Learning
• GP-ConvCNP: Better Generalization for Convolutional Conditional Neural Processes on Time Series Data