Scalable Cross Validation Losses for Gaussian Process Models

Abstract

We introduce a simple and scalable method for training Gaussian process (GP) models that exploits cross-validation and nearest neighbor truncation. To accommodate binary and multi-class classification we leverage Pòlya-Gamma auxiliary variables and variational inference. In an extensive empirical comparison with a number of alternative methods for scalable GP regression and classification, we find that our method offers fast training and excellent predictive performance. We argue that the good predictive performance can be traced to the non-parametric nature of the resulting predictive distributions as well as to the cross-validation loss, which provides robustness against model mis-specification.

arXiv comment20 pages

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Scalable Variational Gaussian Process Classification
• Dependent Multinomial Models Made Easy: Stick Breaking with the Pólya-Gamma Augmentation
• Differentially Private Gaussian Processes
• Towards new cross-validation-based estimators for Gaussian process regression: efficient adjoint computation of gradients
• Sparse Gaussian Processes via Parametric Families of Compactly-supported Kernels

Cited by in corpus (2)

• Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes
• Gaussian Process Inference Using Mini-batch Stochastic Gradient Descent: Convergence Guarantees and Empirical Benefits