Kernel Interpolation for Scalable Online Gaussian Processes

Abstract

Gaussian processes (GPs) provide a gold standard for performance in online settings, such as sample-efficient control and black box optimization, where we need to update a posterior distribution as we acquire data in a sequential fashion. However, updating a GP posterior to accommodate even a single new observation after having observed $n$ points incurs at least $O(n)$ computations in the exact setting. We show how to use structured kernel interpolation to efficiently recycle computations for constant-time $O(1)$ online updates with respect to the number of points $n$, while retaining exact inference. We demonstrate the promise of our approach in a range of online regression and classification settings, Bayesian optimization, and active sampling to reduce error in malaria incidence forecasting. Code is available at https://github.com/wjmaddox/online_gp.

arXiv commentAISTATS 2021

References in corpus (7)

• Gaussian Processes for Big Data
• A Tutorial on Bayesian Optimization
• Gaussian Process Kernels for Pattern Discovery and Extrapolation
• BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization
• Dirichlet-based Gaussian Processes for Large-scale Calibrated Classification
• PBODL : Parallel Bayesian Online Deep Learning for Click-Through Rate Prediction in Tencent Advertising System
• Scalable Gaussian Processes with Grid-Structured Eigenfunctions (GP-GRIEF)