Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition
arXiv:1710.07324
Abstract
We propose a method (TT-GP) for approximate inference in Gaussian Process (GP) models. We build on previous scalable GP research including stochastic variational inference based on inducing inputs, kernel interpolation, and structure exploiting algebra. The key idea of our method is to use Tensor Train decomposition for variational parameters, which allows us to train GPs with billions of inducing inputs and achieve state-of-the-art results on several benchmarks. Further, our approach allows for training kernels based on deep neural networks without any modifications to the underlying GP model. A neural network learns a multidimensional embedding for the data, which is used by the GP to make the final prediction. We train GP and neural network parameters end-to-end without pretraining, through maximization of GP marginal likelihood. We show the efficiency of the proposed approach on several regression and classification benchmark datasets including MNIST, CIFAR-10, and Airline.
References in corpus (1)
Cited by in corpus (12)
- Machine learning and the physical sciences
- When Gaussian Process Meets Big Data: A Review of Scalable GPs
- Exact Gaussian Processes on a Million Data Points
- Kernel methods through the roof: handling billions of points efficiently
- GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning
- Scalable Multi-Task Gaussian Process Tensor Regression for Normative Modeling of Structured Variation in Neuroimaging Data
- Adaptive Learning of Tensor Network Structures
- Beyond the Mean-Field: Structured Deep Gaussian Processes Improve the Predictive Uncertainties
- Scalable Gaussian Process Inference with Finite-data Mean and Variance Guarantees
- The Renyi Gaussian Process: Towards Improved Generalization
- Hierarchical Inducing Point Gaussian Process for Inter-domain Observations
- A Sampling-Based Method for Tensor Ring Decomposition