Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)

Abstract

We introduce a new structured kernel interpolation (SKI) framework, which generalises and unifies inducing point methods for scalable Gaussian processes (GPs). SKI methods produce kernel approximations for fast computations through kernel interpolation. The SKI framework clarifies how the quality of an inducing point approach depends on the number of inducing (aka interpolation) points, interpolation strategy, and GP covariance kernel. SKI also provides a mechanism to create new scalable kernel methods, through choosing different kernel interpolation strategies. Using SKI, with local cubic kernel interpolation, we introduce KISS-GP, which is 1) more scalable than inducing point alternatives, 2) naturally enables Kronecker and Toeplitz algebra for substantial additional gains in scalability, without requiring any grid data, and 3) can be used for fast and expressive kernel learning. KISS-GP costs O(n) time and storage for GP inference. We evaluate KISS-GP for kernel matrix approximation, kernel learning, and natural sound modelling.

19 pages, 4 figures

References in corpus (3)

• Gaussian Processes for Big Data
• Gaussian Process Kernels for Pattern Discovery and Extrapolation
• How to Scale Up Kernel Methods to Be As Good As Deep Neural Nets

Cited by in corpus (76)

• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Gaussian process regression for forecasting battery state of health
• Neural Architecture Search with Bayesian Optimisation and Optimal Transport
• When Gaussian Process Meets Big Data: A Review of Scalable GPs
• Local approximate Gaussian process regression for data-driven constitutive laws: Development and comparison with neural networks
• pySOT and POAP: An event-driven asynchronous framework for surrogate optimization
• A scalable end-to-end Gaussian process adapter for irregularly sampled time series classification
• Structured Bayesian Gaussian process latent variable model: applications to data-driven dimensionality reduction and high-dimensional inversion
• Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance
• GPLaSDI: Gaussian Process-based Interpretable Latent Space Dynamics Identification through Deep Autoencoder
• Exact Gaussian Processes on a Million Data Points
• Kernel methods through the roof: handling billions of points efficiently
• A literature survey of matrix methods for data science
• Preconditioning Kernel Matrices
• Adaptive Deep Kernel Learning
• Deep Kernel Learning
• Causal Inference for Time series Analysis: Problems, Methods and Evaluation
• Constant-Time Predictive Distributions for Gaussian Processes
• Deep Structured Mixtures of Gaussian Processes
• Randomly Projected Additive Gaussian Processes for Regression
• Sequential Gaussian Processes for Online Learning of Nonstationary Functions
• Scalable Gaussian Processes on Discrete Domains
• Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization
• Efficient force field and energy emulation through partition of permutationally equivalent atoms
• Scaling Gaussian Processes with Derivative Information Using Variational Inference
• Deep Bayesian Gaussian Processes for Uncertainty Estimation in Electronic Health Records
• Discovery of structure-property relations for molecules via hypothesis-driven active learning over the chemical space
• Learning the kernel matrix via predictive low-rank approximations
• Large-scale Heteroscedastic Regression via Gaussian Process
• Fidelity-Weighted Learning
• Informative Planning for Worst-Case Error Minimisation in Sparse Gaussian Process Regression
• Sparse Orthogonal Variational Inference for Gaussian Processes
• Fast increased fidelity approximate Gibbs samplers for Bayesian Gaussian process regression
• Learning to Guide Random Search
• Modeling Dynamic Functional Connectivity with Latent Factor Gaussian Processes
• Spatio-Temporal Variational Gaussian Processes
• Distilled Thompson Sampling: Practical and Efficient Thompson Sampling via Imitation Learning
• Near-linear Time Gaussian Process Optimization with Adaptive Batching and Resparsification
• Gauss-Legendre Features for Gaussian Process Regression
• Fast Kronecker Matrix-Matrix Multiplication on GPUs
• Recent Advances in Data-Driven Wireless Communication Using Gaussian Processes: A Comprehensive Survey
• Kernelized Capsule Networks
• SKIing on Simplices: Kernel Interpolation on the Permutohedral Lattice for Scalable Gaussian Processes
• Scalable Gaussian Process Classification with Additive Noise for Various Likelihoods
• Inter-domain Deep Gaussian Processes
• Kernel Conditional Density Operators
• Quadruply Stochastic Gaussian Processes
• Sparse Gaussian Processes via Parametric Families of Compactly-supported Kernels
• Scalable Gaussian Process Inference with Finite-data Mean and Variance Guarantees
• Deep Coregionalization for the Emulation of Spatial-Temporal Fields
• SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives
• Gaussian Process Inference Using Mini-batch Stochastic Gradient Descent: Convergence Guarantees and Empirical Benefits
• Input Dependent Sparse Gaussian Processes
• Mixed Variable Bayesian Optimization with Frequency Modulated Kernels
• End-to-End Probabilistic Inference for Nonstationary Audio Analysis
• Deep Kernel Learning for Mortality Prediction in the Face of Temporal Shift
• Large-scale magnetic field maps using structured kernel interpolation for Gaussian process regression
• Deep Latent-Variable Kernel Learning
• UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Data
• MBORE: Multi-objective Bayesian Optimisation by Density-Ratio Estimation
• Quantifying Predictive Uncertainty in Medical Image Analysis with Deep Kernel Learning
• Scalable Gaussian Process Regression for Kernels with a Non-Stationary Phase
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition
• The Fast Kernel Transform
• Scalable Hyperparameter Optimization with Lazy Gaussian Processes
• Modulating Scalable Gaussian Processes for Expressive Statistical Learning
• Robust and Scalable SDE Learning: A Functional Perspective
• Scalable Grouped Gaussian Processes via Direct Cholesky Functional Representations
• Bayesian task embedding for few-shot Bayesian optimization
• Healing Products of Gaussian Processes
• Linear-time inference for Gaussian Processes on one dimension
• Adaptive Cholesky Gaussian Processes
• The Statistical Cost of Robust Kernel Hyperparameter Tuning
• Faster Kernel Interpolation for Gaussian Processes
• Gaussian Processes with Skewed Laplace Spectral Mixture Kernels for Long-term Forecasting
• Sparse within Sparse Gaussian Processes using Neighbor Information