Compositionally-Warped Gaussian Processes

Abstract

The Gaussian process (GP) is a nonparametric prior distribution over functions indexed by time, space, or other high-dimensional index set. The GP is a flexible model yet its limitation is given by its very nature: it can only model Gaussian marginal distributions. To model non-Gaussian data, a GP can be warped by a nonlinear transformation (or warping) as performed by warped GPs (WGPs) and more computationally-demanding alternatives such as Bayesian WGPs and deep GPs. However, the WGP requires a numerical approximation of the inverse warping for prediction, which increases the computational complexity in practice. To sidestep this issue, we construct a novel class of warpings consisting of compositions of multiple elementary functions, for which the inverse is known explicitly. We then propose the compositionally-warped GP (CWGP), a non-Gaussian generative model whose expressiveness follows from its deep compositional architecture, and its computational efficiency is guaranteed by the analytical inverse warping. Experimental validation using synthetic and real-world datasets confirms that the proposed CWGP is robust to the choice of warpings and provides more accurate point predictions, better trained models and shorter computation times than WGP.

Accepted at Elsevier Neural Networks, DOI added and author order corrected

References in corpus (7)

• Deep Learning in Neural Networks: An Overview
• Variational Inference with Normalizing Flows
• Structure Discovery in Nonparametric Regression through Compositional Kernel Search
• Gaussian Process Kernels for Pattern Discovery and Extrapolation
• An introduction to sampling via measure transport
• Bayesian Nonparametric Spectral Estimation
• Recovering Latent Signals from a Mixture of Measurements using a Gaussian Process Prior

Cited by in corpus (7)

• Transforming Gaussian Processes With Normalizing Flows
• Monte Carlo inference for semiparametric Bayesian regression
• Transport Gaussian Processes for Regression
• Recent Advances in Data-Driven Wireless Communication Using Gaussian Processes: A Comprehensive Survey
• Transformation-based generalized spatial regression using the spmoran package: Case study examples
• State-space deep Gaussian processes with applications
• Compositionally-warped additive mixed modeling for a wide variety of non-Gaussian spatial data