Stochastic expansions using continuous dictionaries: Lévy adaptive regression kernels

Abstract

This article describes a new class of prior distributions for nonparametric function estimation. The unknown function is modeled as a limit of weighted sums of kernels or generator functions indexed by continuous parameters that control local and global features such as their translation, dilation, modulation and shape. Lévy random fields and their stochastic integrals are employed to induce prior distributions for the unknown functions or, equivalently, for the number of kernels and for the parameters governing their features. Scaling, shape, and other features of the generating functions are location-specific to allow quite different function properties in different parts of the space, as with wavelet bases and other methods employing overcomplete dictionaries. We provide conditions under which the stochastic expansions converge in specified Besov or Sobolev norms. Under a Gaussian error model, this may be viewed as a sparse regression problem, with regularization induced via the Lévy random field prior distribution. Posterior inference for the unknown functions is based on a reversible jump Markov chain Monte Carlo algorithm. We compare the Lévy Adaptive Regression Kernel (LARK) method to wavelet-based methods using some of the standard test functions, and illustrate its flexibility and adaptability in nonstationary applications.

Published in at http://dx.doi.org/10.1214/11-AOS889 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• Convergence rates of posterior distributions for noniid observations
• The Use of Unlabeled Data in Predictive Modeling

Cited by in corpus (15)

• Beta-Negative Binomial Process and Poisson Factor Analysis
• Locally adaptive factor processes for multivariate time series
• Nonparametric Bayesian Factor Analysis for Dynamic Count Matrices
• Bayesian nonparametric models for peak identification in MALDI-TOF mass spectroscopy
• Augment-and-Conquer Negative Binomial Processes
• Softplus Regressions and Convex Polytopes
• Nonparametric Bayesian Negative Binomial Factor Analysis
• Nonstationary, Nonparametric, Nonseparable Bayesian Spatio-Temporal Modeling Using Kernel Convolution of Order Based Dependent Dirichlet Process
• Gradient Flow from a Random Walk in Hilbert Space
• Scalable Lévy Process Priors for Spectral Kernel Learning
• Locally Adaptive Bayes Nonparametric Regression via Nested Gaussian Processes
• Bayesian Levy-Dynamic Spatio-Temporal Process: Towards Big Data Analysis
• Local shrinkage rules, Levy processes, and regularized regression
• Multiscale Shrinkage and Lévy Processes
• Generalized Negative Binomial Processes and the Representation of Cluster Structures