A Tutorial on Sparse Gaussian Processes and Variational Inference

Abstract

Gaussian processes (GPs) provide a framework for Bayesian inference that can offer principled uncertainty estimates for a large range of problems. For example, if we consider regression problems with Gaussian likelihoods, a GP model enjoys a posterior in closed form. However, identifying the posterior GP scales cubically with the number of training examples and requires to store all examples in memory. In order to overcome these obstacles, sparse GPs have been proposed that approximate the true posterior GP with pseudo-training examples. Importantly, the number of pseudo-training examples is user-defined and enables control over computational and memory complexity. In the general case, sparse GPs do not enjoy closed-form solutions and one has to resort to approximate inference. In this context, a convenient choice for approximate inference is variational inference (VI), where the problem of Bayesian inference is cast as an optimization problem -- namely, to maximize a lower bound of the log marginal likelihood. This paves the way for a powerful and versatile framework, where pseudo-training examples are treated as optimization arguments of the approximate posterior that are jointly identified together with hyperparameters of the generative model (i.e. prior and likelihood). The framework can naturally handle a wide scope of supervised learning problems, ranging from regression with heteroscedastic and non-Gaussian likelihoods to classification problems with discrete labels, but also problems with multidimensional labels. The purpose of this tutorial is to provide access to the basic matter for readers without prior knowledge in both GPs and VI. A proper exposition to the subject enables also access to more recent advances (like importance-weighted VI as well as interdomain, multioutput and deep GPs) that can serve as an inspiration for new research ideas.

References in corpus (37)

• Auto-Encoding Variational Bayes
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Trust Region Policy Optimization
• Soft Actor-Critic Algorithms and Applications
• Semi-Supervised Learning with Deep Generative Models
• Weight Uncertainty in Neural Networks
• Gaussian Processes for Big Data
• Reinforcement Learning with Deep Energy-Based Policies
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
• Scalable Variational Gaussian Process Classification
• Model-Ensemble Trust-Region Policy Optimization
• Benchmarking Model-Based Reinforcement Learning
• Bridging the Gap Between Value and Policy Based Reinforcement Learning
• Equivalence Between Policy Gradients and Soft Q-Learning
• Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control
• Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees
• A unified view of entropy-regularized Markov decision processes
• Variational Fourier features for Gaussian processes
• Avoiding pathologies in very deep networks
• Bayesian Layers: A Module for Neural Network Uncertainty
• Exploring Model-based Planning with Policy Networks
• MCMC for Variationally Sparse Gaussian Processes
• Importance Weighting and Variational Inference
• Gaussian Process Conditional Density Estimation
• Information-Theoretic Bounded Rationality
• Practical Hilbert space approximate Bayesian Gaussian processes for probabilistic programming
• A Unified Bellman Equation for Causal Information and Value in Markov Decision Processes
• Deep Gaussian Processes with Importance-Weighted Variational Inference
• A Unified Bellman Optimality Principle Combining Reward Maximization and Empowerment
• An Information-Theoretic Optimality Principle for Deep Reinforcement Learning
• Sparse Gaussian Processes with Spherical Harmonic Features
• Disentangled Skill Embeddings for Reinforcement Learning
• Bounded Rational Decision-Making in Feedforward Neural Networks
• Variational Orthogonal Features
• Model-Based Regularization for Deep Reinforcement Learning with Transcoder Networks
• Variational Inference for Data-Efficient Model Learning in POMDPs
• Doubly Sparse Variational Gaussian Processes

Cited by in corpus (7)

• Nonnegative spatial factorization
• GPflux: A Library for Deep Gaussian Processes
• Sparse Uncertainty Representation in Deep Learning with Inducing Weights
• Hybrid Bayesian Neural Networks with Functional Probabilistic Layers
• Deep Neural Networks as Point Estimates for Deep Gaussian Processes
• Bellman: A Toolbox for Model-Based Reinforcement Learning in TensorFlow
• Gaussian Processes on Hypergraphs