Meta-Learning Mean Functions for Gaussian Processes

Abstract

When fitting Bayesian machine learning models on scarce data, the main challenge is to obtain suitable prior knowledge and encode it into the model. Recent advances in meta-learning offer powerful methods for extracting such prior knowledge from data acquired in related tasks. When it comes to meta-learning in Gaussian process models, approaches in this setting have mostly focused on learning the kernel function of the prior, but not on learning its mean function. In this work, we explore meta-learning the mean function of a Gaussian process prior. We present analytical and empirical evidence that mean function learning can be useful in the meta-learning setting, discuss the risk of overfitting, and draw connections to other meta-learning approaches, such as model agnostic meta-learning and functional PCA.

References in corpus (10)

• Deep Learning in Neural Networks: An Overview
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Theoretical Models of Learning to Learn
• GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
• Conditional Neural Processes
• Improving Output Uncertainty Estimation and Generalization in Deep Learning via Neural Network Gaussian Processes
• Understanding symmetries in deep networks
• Deep Kernel Learning
• Scalable Gaussian Processes on Discrete Domains

Cited by in corpus (8)

• MGP-AttTCN: An Interpretable Machine Learning Model for the Prediction of Sepsis
• What do you Mean? The Role of the Mean Function in Bayesian Optimisation
• Principal component analysis for Gaussian process posteriors
• Non-Gaussian Gaussian Processes for Few-Shot Regression
• Few-shot Learning for Spatial Regression
• End-to-End Learning of Deep Kernel Acquisition Functions for Bayesian Optimization
• Bayesian Meta-Learning Through Variational Gaussian Processes
• Transfer Bayesian Meta-learning via Weighted Free Energy Minimization