Understanding Probabilistic Sparse Gaussian Process Approximations

Abstract

Good sparse approximations are essential for practical inference in Gaussian Processes as the computational cost of exact methods is prohibitive for large datasets. The Fully Independent Training Conditional (FITC) and the Variational Free Energy (VFE) approximations are two recent popular methods. Despite superficial similarities, these approximations have surprisingly different theoretical properties and behave differently in practice. We thoroughly investigate the two methods for regression both analytically and through illustrative examples, and draw conclusions to guide practical application.

published in Advances in Neural Information Processing Systems 29 (NIPS 2016)

References in corpus (1)

• Scalable Variational Gaussian Process Classification

Cited by in corpus (15)

• Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
• A Unifying Framework for Gaussian Process Pseudo-Point Approximations using Power Expectation Propagation
• Deep Gaussian Processes for Biogeophysical Parameter Retrieval and Model Inversion
• Streaming Sparse Gaussian Process Approximations
• Variational Inference for Gaussian Process Models with Linear Complexity
• On Random Subsampling of Gaussian Process Regression: A Graphon-Based Analysis
• Transforming Gaussian Processes With Normalizing Flows
• Deep Gaussian Processes for geophysical parameter retrieval
• Scalable Multi-Class Gaussian Process Classification using Expectation Propagation
• Fast Deep Mixtures of Gaussian Process Experts
• Spatio-Temporal Variational Gaussian Processes
• Standing Wave Decomposition Gaussian Process
• Structured Variational Inference for Coupled Gaussian Processes
• Scalable GAM using sparse variational Gaussian processes
• Asynchronous Distributed Variational Gaussian Processes for Regression