Scalable Log Determinants for Gaussian Process Kernel Learning

Abstract

For applications as varied as Bayesian neural networks, determinantal point processes, elliptical graphical models, and kernel learning for Gaussian processes (GPs), one must compute a log determinant of an $n\times n$ positive definite matrix, and its derivatives - leading to prohibitive $\mathcal{O}(n^3)$ computations. We propose novel $\mathcal{O}(n)$ approaches to estimating these quantities from only fast matrix vector multiplications (MVMs). These stochastic approximations are based on Chebyshev, Lanczos, and surrogate models, and converge quickly even for kernel matrices that have challenging spectra. We leverage these approximations to develop a scalable Gaussian process approach to kernel learning. We find that Lanczos is generally superior to Chebyshev for kernel learning, and that a surrogate approach can be highly efficient and accurate with popular kernels.

Appears at Advances in Neural Information Processing Systems 30 (NIPS), 2017

References in corpus (1)

• Determinantal point processes for machine learning

Cited by in corpus (13)

• GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
• Product Kernel Interpolation for Scalable Gaussian Processes
• Scaling Gaussian Process Regression with Derivatives
• MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning
• Constant-Time Predictive Distributions for Gaussian Processes
• Iterative Methods for Vecchia-Laplace Approximations for Latent Gaussian Process Models
• VBALD - Variational Bayesian Approximation of Log Determinants
• Accurate Kernel Learning for Linear Gaussian Markov Processes using a Scalable Likelihood Computation
• Sparse inversion for derivative of log determinant
• Scalable Bayesian Optimization with Sparse Gaussian Process Models
• Surveillance Evasion Through Bayesian Reinforcement Learning
• Implicit Riemannian Concave Potential Maps
• Conjugate Gradients for Kernel Machines