Pathwise Conditioning of Gaussian Processes
arXiv:2011.04026
Abstract
As Gaussian processes are used to answer increasingly complex questions, analytic solutions become scarcer and scarcer. Monte Carlo methods act as a convenient bridge for connecting intractable mathematical expressions with actionable estimates via sampling. Conventional approaches for simulating Gaussian process posteriors view samples as draws from marginal distributions of process values at finite sets of input locations. This distribution-centric characterization leads to generative strategies that scale cubically in the size of the desired random vector. These methods are prohibitively expensive in cases where we would, ideally, like to draw high-dimensional vectors or even continuous sample paths. In this work, we investigate a different line of reasoning: rather than focusing on distributions, we articulate Gaussian conditionals at the level of random variables. We show how this pathwise interpretation of conditioning gives rise to a general family of approximations that lend themselves to efficiently sampling Gaussian process posteriors. Starting from first principles, we derive these methods and analyze the approximation errors they introduce. We, then, ground these results by exploring the practical implications of pathwise conditioning in various applied settings, such as global optimization and reinforcement learning.
References in corpus (20)
- Practical Bayesian Optimization of Machine Learning Algorithms
- Gaussian Processes for Data-Efficient Learning in Robotics and Control
- GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
- Gaussian Processes for Big Data
- A Tutorial on Bayesian Optimization
- Scalable Variational Gaussian Process Classification
- Sequential design of computer experiments for the estimation of a probability of failure
- Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
- Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control
- Batched Large-scale Bayesian Optimization in High-dimensional Spaces
- Variational Fourier features for Gaussian processes
- A Framework for Interdomain and Multioutput Gaussian Processes
- Maximizing acquisition functions for Bayesian optimization
- On the Error of Random Fourier Features
- Matérn Gaussian processes on Riemannian manifolds
- Orthogonally Decoupled Variational Gaussian Processes
- Deep learning with differential Gaussian process flows
- Banded Matrix Operators for Gaussian Markov Models in the Automatic Differentiation Era
- Know Your Boundaries: Constraining Gaussian Processes by Variational Harmonic Features
- Matérn Gaussian Processes on Graphs