Gaussian Process Kernels for Pattern Discovery and Extrapolation

Abstract

Gaussian processes are rich distributions over functions, which provide a Bayesian nonparametric approach to smoothing and interpolation. We introduce simple closed form kernels that can be used with Gaussian processes to discover patterns and enable extrapolation. These kernels are derived by modelling a spectral density -- the Fourier transform of a kernel -- with a Gaussian mixture. The proposed kernels support a broad class of stationary covariances, but Gaussian process inference remains simple and analytic. We demonstrate the proposed kernels by discovering patterns and performing long range extrapolation on synthetic examples, as well as atmospheric CO2 trends and airline passenger data. We also show that we can reconstruct standard covariances within our framework.

10 pages, 5 figures, 1 table. Minor edits and titled changed from "Gaussian Process Covariance Kernels for Pattern Discovery and Extrapolation" to "Gaussian Process Kernels for Pattern Discovery and Extrapolation". Appears at the International Conference on Machine Learning (ICML), JMLR W&CP 28(3):1067-1075, 2013

Cited by in corpus (77)

• Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)
• Student-t Processes as Alternatives to Gaussian Processes
• How priors of initial hyperparameters affect Gaussian process regression models
• Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels
• Physics-informed machine learning for Structural Health Monitoring
• Linear Multiple Low-Rank Kernel Based Stationary Gaussian Processes Regression for Time Series
• A scalable end-to-end Gaussian process adapter for irregularly sampled time series classification
• Compositionally-Warped Gaussian Processes
• Differentiable Compositional Kernel Learning for Gaussian Processes
• Exact Gaussian Processes on a Million Data Points
• A Nonparametric Bayesian Methodology for Regression Discontinuity Designs
• Time series forecasting with Gaussian Processes needs priors
• Generalized Optimal Matching Methods for Causal Inference
• Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond
• Subspace Inference for Bayesian Deep Learning
• Gaussian process regression for forest attribute estimation from airborne laser scanning data
• Gaussian Process Prior Variational Autoencoders
• Non-separable Covariance Kernels for Spatiotemporal Gaussian Processes based on a Hybrid Spectral Method and the Harmonic Oscillator
• A la Carte - Learning Fast Kernels
• Deep Kernel Learning
• Causal Inference for Time series Analysis: Problems, Methods and Evaluation
• Constant-Time Predictive Distributions for Gaussian Processes
• On Local Optimizers of Acquisition Functions in Bayesian Optimization
• Expressive Priors in Bayesian Neural Networks: Kernel Combinations and Periodic Functions
• Automated Spectral Kernel Learning
• A Gaussian Process perspective on Convolutional Neural Networks
• Randomly Projected Additive Gaussian Processes for Regression
• Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization
• Benchmarking the Performance of Bayesian Optimization across Multiple Experimental Materials Science Domains
• GPatt: Fast Multidimensional Pattern Extrapolation with Gaussian Processes
• Discovery of structure-property relations for molecules via hypothesis-driven active learning over the chemical space
• More robust estimation of sample average treatment effects using Kernel Optimal Matching in an observational study of spine surgical interventions
• Learning the kernel matrix via predictive low-rank approximations
• Few-Shot Bayesian Optimization with Deep Kernel Surrogates
• Physiological Gaussian Process Priors for the Hemodynamics in fMRI Analysis
• Fourier Sparse Leverage Scores and Approximate Kernel Learning
• Non-Gaussian Gaussian Processes for Few-Shot Regression
• PLSO: A generative framework for decomposing nonstationary time-series into piecewise stationary oscillatory components
• Kernel Interpolation for Scalable Online Gaussian Processes
• Bayesian Optimization Algorithms for Accelerator Physics
• Marginalised Gaussian Processes with Nested Sampling
• Transport Gaussian Processes for Regression
• Identifiability of Covariance Kernels in the Gaussian Process Regression Model
• Deep Recurrent Gaussian Process with Variational Sparse Spectrum Approximation
• Recent Advances in Data-Driven Wireless Communication Using Gaussian Processes: A Comprehensive Survey
• Harmonized Multimodal Learning with Gaussian Process Latent Variable Models
• On Learning the Transformer Kernel
• Evolving Gaussian Process kernels from elementary mathematical expressions
• Neural Likelihoods for Multi-Output Gaussian Processes
• Gaussian Process Uniform Error Bounds with Unknown Hyperparameters for Safety-Critical Applications
• Efficient Phase Diagram Sampling by Active Learning
• Learning Nonparametric Volterra Kernels with Gaussian Processes
• Marginalising over Stationary Kernels with Bayesian Quadrature
• Robust non-parametric mortality and fertility modelling and forecasting: Gaussian process regression approaches
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition
• Conditional Deep Gaussian Processes: multi-fidelity kernel learning
• KernelNet: A Data-Dependent Kernel Parameterization for Deep Generative Modeling
• Deep kernel learning for integral measurements
• Learning Compositional Sparse Gaussian Processes with a Shrinkage Prior
• Latent variable modeling with random features
• End-to-End Probabilistic Inference for Nonstationary Audio Analysis
• A Probabilistic Approach to Nonparametric Local Volatility
• Gaussian Processes for Analyzing Positioned Trajectories in Sports
• String Gaussian Process Kernels
• Optimally adaptive Bayesian spectral density estimation for stationary and nonstationary processes
• Scalable Hybrid HMM with Gaussian Process Emission for Sequential Time-series Data Clustering
• Spectrum Gaussian Processes Based On Tunable Basis Functions
• A Self-Supervised Framework for Function Learning and Extrapolation
• Black-box density function estimation using recursive partitioning
• Faster Kernel Interpolation for Gaussian Processes
• Modeling Regime Shifts in Multiple Time Series
• Gaussian Processes with Skewed Laplace Spectral Mixture Kernels for Long-term Forecasting
• The Minecraft Kernel: Modelling correlated Gaussian Processes in the Fourier domain
• The Statistical Cost of Robust Kernel Hyperparameter Tuning
• Linear-time inference for Gaussian Processes on one dimension
• Partial sequence labeling with structured Gaussian Processes
• Learning Data-adaptive Nonparametric Kernels