MCMC for Variationally Sparse Gaussian Processes

Abstract

Gaussian process (GP) models form a core part of probabilistic machine learning. Considerable research effort has been made into attacking three issues with GP models: how to compute efficiently when the number of data is large; how to approximate the posterior when the likelihood is not Gaussian and how to estimate covariance function parameter posteriors. This paper simultaneously addresses these, using a variational approximation to the posterior which is sparse in support of the function but otherwise free-form. The result is a Hybrid Monte-Carlo sampling scheme which allows for a non-Gaussian approximation over the function values and covariance parameters simultaneously, with efficient computations based on inducing-point sparse GPs. Code to replicate each experiment in this paper will be available shortly.

16 pages

References in corpus (7)

• Scalable Variational Gaussian Process Classification
• Elliptical slice sampling
• Slice sampling covariance hyperparameters of latent Gaussian models
• Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models
• Variational Inference for Gaussian Process Modulated Poisson Processes
• Scalable Nonparametric Bayesian Inference on Point Processes with Gaussian Processes
• Enabling scalable stochastic gradient-based inference for Gaussian processes by employing the Unbiased LInear System SolvEr (ULISSE)

Cited by in corpus (25)

• When Gaussian Process Meets Big Data: A Review of Scalable GPs
• Random Feature Expansions for Deep Gaussian Processes
• Variational Fourier features for Gaussian processes
• A Framework for Interdomain and Multioutput Gaussian Processes
• Multi-fidelity classification using Gaussian processes: accelerating the prediction of large-scale computational models
• Approximate Inference for Fully Bayesian Gaussian Process Regression
• Bayesian Semi-supervised Learning with Graph Gaussian Processes
• A Tutorial on Sparse Gaussian Processes and Variational Inference
• Accelerating Monte Carlo Bayesian Inference via Approximating Predictive Uncertainty over Simplex
• Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation
• Parametric Gaussian Process Regressors
• Probabilistic selection of inducing points in sparse Gaussian processes
• Sparse Orthogonal Variational Inference for Gaussian Processes
• Calibrated simplex-mapping classification
• Non-parametric generalized linear model
• Stein Variational Gaussian Processes
• Large-Scale Cox Process Inference using Variational Fourier Features
• Scalable Gaussian Process Classification with Additive Noise for Various Likelihoods
• SKIing on Simplices: Kernel Interpolation on the Permutohedral Lattice for Scalable Gaussian Processes
• Input Dependent Sparse Gaussian Processes
• On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approach
• Regularized Sparse Gaussian Processes
• Hierarchical Inducing Point Gaussian Process for Inter-domain Observations
• Conditionally Independent Multiresolution Gaussian Processes
• Likelihood-Free Gaussian Process for Regression