SKIing on Simplices: Kernel Interpolation on the Permutohedral Lattice for Scalable Gaussian Processes

Abstract

State-of-the-art methods for scalable Gaussian processes use iterative algorithms, requiring fast matrix vector multiplies (MVMs) with the covariance kernel. The Structured Kernel Interpolation (SKI) framework accelerates these MVMs by performing efficient MVMs on a grid and interpolating back to the original space. In this work, we develop a connection between SKI and the permutohedral lattice used for high-dimensional fast bilateral filtering. Using a sparse simplicial grid instead of a dense rectangular one, we can perform GP inference exponentially faster in the dimension than SKI. Our approach, Simplex-GP, enables scaling SKI to high dimensions, while maintaining strong predictive performance. We additionally provide a CUDA implementation of Simplex-GP, which enables significant GPU acceleration of MVM based inference.

arXiv commentInternational Conference on Machine Learning (ICML), 2021

References in corpus (10)

• Language Models are Few-Shot Learners
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Gaussian Processes for Big Data
• Scalable Variational Gaussian Process Classification
• Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)
• Kernel Operations on the GPU, with Autodiff, without Memory Overflows
• Thoughts on Massively Scalable Gaussian Processes
• MCMC for Variationally Sparse Gaussian Processes
• Permutohedral Lattice CNNs
• Bias-Free Scalable Gaussian Processes via Randomized Truncations