Stochastic approximation of score functions for Gaussian processes

Abstract

We discuss the statistical properties of a recently introduced unbiased stochastic approximation to the score equations for maximum likelihood calculation for Gaussian processes. Under certain conditions, including bounded condition number of the covariance matrix, the approach achieves $O(n)$ storage and nearly $O(n)$ computational effort per optimization step, where $n$ is the number of data sites. Here, we prove that if the condition number of the covariance matrix is bounded, then the approximate score equations are nearly optimal in a well-defined sense. Therefore, not only is the approximation efficient to compute, but it also has comparable statistical properties to the exact maximum likelihood estimates. We discuss a modification of the stochastic approximation in which design elements of the stochastic terms mimic patterns from a $2^n$ factorial design. We prove these designs are always at least as good as the unstructured design, and we demonstrate through simulation that they can produce a substantial improvement over random designs. Our findings are validated by numerical experiments on simulated data sets of up to 1 million observations. We apply the approach to fit a space-time model to over 80,000 observations of total column ozone contained in the latitude band $40^{\circ }$-$50^{\circ }$N during April 2012.

Published in at http://dx.doi.org/10.1214/13-AOAS627 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (1)

• Spatial variation of total column ozone on a global scale

Cited by in corpus (9)

• A multi-resolution approximation for massive spatial datasets
• Fast spatial Gaussian process maximum likelihood estimation via skeletonization factorizations
• Parallel Approximation of the Maximum Likelihood Estimation for the Prediction of Large-Scale Geostatistics Simulations
• On Approximation for Fractional Stochastic Partial Differential Equations on the Sphere
• Multi-Level Restricted Maximum Likelihood Covariance Estimation and Kriging for Large Non-Gridded Spatial Datasets
• Estimating Atmospheric Motion Winds from Satellite Image Data using Space-time Drift Models
• Multilevel Stochastic Optimization for Imputation in Massive Medical Data Records
• Iterative Methods for Vecchia-Laplace Approximations for Latent Gaussian Process Models
• The inverse Kalman filter