Recyclable Gaussian Processes

Abstract

We present a new framework for recycling independent variational approximations to Gaussian processes. The main contribution is the construction of variational ensembles given a dictionary of fitted Gaussian processes without revisiting any subset of observations. Our framework allows for regression, classification and heterogeneous tasks, i.e. mix of continuous and discrete variables over the same input domain. We exploit infinite-dimensional integral operators based on the Kullback-Leibler divergence between stochastic processes to re-combine arbitrary amounts of variational sparse approximations with different complexity, likelihood model and location of the pseudo-inputs. Extensive results illustrate the usability of our framework in large-scale distributed experiments, also compared with the exact inference models in the literature.

References in corpus (7)

• Gaussian Processes for Big Data
• Scalable Variational Gaussian Process Classification
• Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions
• Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models
• Private Federated Learning with Domain Adaptation
• Partitioned Variational Inference: A unified framework encompassing federated and continual learning
• Continual Multi-task Gaussian Processes