Scalable Training of Inference Networks for Gaussian-Process Models
arXiv:1905.10969
Abstract
Inference in Gaussian process (GP) models is computationally challenging for large data, and often difficult to approximate with a small number of inducing points. We explore an alternative approximation that employs stochastic inference networks for a flexible inference. Unfortunately, for such networks, minibatch training is difficult to be able to learn meaningful correlations over function outputs for a large dataset. We propose an algorithm that enables such training by tracking a stochastic, functional mirror-descent algorithm. At each iteration, this only requires considering a finite number of input locations, resulting in a scalable and easy-to-implement algorithm. Empirical results show comparable and, sometimes, superior performance to existing sparse variational GP methods.
arXiv commentICML 2019. Update results added in the camera-ready version
References in corpus (14)
- Neural Tangent Kernel: Convergence and Generalization in Neural Networks
- Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
- Scalable Bayesian Optimization Using Deep Neural Networks
- Bayesian Dark Knowledge
- Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
- Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes
- Functional Variational Bayesian Neural Networks
- Variational Fourier features for Gaussian processes
- Provable Bayesian Inference via Particle Mirror Descent
- ZhuSuan: A Library for Bayesian Deep Learning
- Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs
- Function Space Particle Optimization for Bayesian Neural Networks
- Variational Implicit Processes
- Analysis and Probability on Infinite-Dimensional Spaces