Probabilistic selection of inducing points in sparse Gaussian processes
arXiv:2010.09370
Abstract
Sparse Gaussian processes and various extensions thereof are enabled through inducing points, that simultaneously bottleneck the predictive capacity and act as the main contributor towards model complexity. However, the number of inducing points is generally not associated with uncertainty which prevents us from applying the apparatus of Bayesian reasoning for identifying an appropriate trade-off. In this work we place a point process prior on the inducing points and approximate the associated posterior through stochastic variational inference. By letting the prior encourage a moderate number of inducing points, we enable the model to learn which and how many points to utilise. We experimentally show that fewer inducing points are preferred by the model as the points become less informative, and further demonstrate how the method can be employed in deep Gaussian processes and latent variable modelling.
arXiv comment37th Conference on Uncertainty in Artificial Intelligence (UAI 2021)
References in corpus (14)
- Adam: A Method for Stochastic Optimization
- Practical Bayesian Optimization of Machine Learning Algorithms
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
- Gaussian Processes for Big Data
- Scalable Variational Gaussian Process Classification
- Variational Fourier features for Gaussian processes
- Avoiding pathologies in very deep networks
- MCMC for Variationally Sparse Gaussian Processes
- Nested Variational Compression in Deep Gaussian Processes
- Orthogonally Decoupled Variational Gaussian Processes
- Multi-resolution Multi-task Gaussian Processes
- Deep Gaussian Processes with Decoupled Inducing Inputs
- Sparse Gaussian Processes Revisited: Bayesian Approaches to Inducing-Variable Approximations