Adaptive Bayesian density regression for high-dimensional data

Abstract

Density regression provides a flexible strategy for modeling the distribution of a response variable $Y$ given predictors $\mathbf{X}=(X_1,\dots ,X_p)$ by letting that the conditional density of $Y$ given $\mathbf{X}$ as a completely unknown function and allowing its shape to change with the value of $\mathbf{X}$. The number of predictors $p$ may be very large, possibly much larger than the number of observations $n$, but the conditional density is assumed to depend only on a much smaller number of predictors, which are unknown. In addition to estimation, the goal is also to select the important predictors which actually affect the true conditional density. We consider a nonparametric Bayesian approach to density regression by constructing a random series prior based on tensor products of spline functions. The proposed prior also incorporates the issue of variable selection. We show that the posterior distribution of the conditional density contracts adaptively at the truth nearly at the optimal oracle rate, determined by the unknown sparsity and smoothness levels, even in the ultra high-dimensional settings where $p$ increases exponentially with $n$. The result is also extended to the anisotropic case where the degree of smoothness can vary in different directions, and both random and deterministic predictors are considered. We also propose a technique to calculate posterior moments of the conditional density function without requiring Markov chain Monte Carlo methods.

Published at http://dx.doi.org/10.3150/14-BEJ663 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)

References in corpus (7)

• Variable selection in nonparametric additive models
• High-dimensional generalized linear models and the lasso
• Rates of contraction of posterior distributions based on Gaussian process priors
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Convergence rates of posterior distributions for noniid observations
• Posterior convergence rates of Dirichlet mixtures at smooth densities
• Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities

Cited by in corpus (1)

• Supremum Norm Posterior Contraction and Credible Sets for Nonparametric Multivariate Regression