An automatic robust Bayesian approach to principal component regression

Abstract

Principal component regression uses principal components as regressors. It is particularly useful in prediction settings with high-dimensional covariates. The existing literature treating of Bayesian approaches is relatively sparse. We introduce a Bayesian approach that is robust to outliers in both the dependent variable and the covariates. Outliers can be thought of as observations that are not in line with the general trend. The proposed approach automatically penalises these observations so that their impact on the posterior gradually vanishes as they move further and further away from the general trend, corresponding to a concept in Bayesian statistics called whole robustness. The predictions produced are thus consistent with the bulk of the data. The approach also exploits the geometry of principal components to efficiently identify those that are significant. Individual predictions obtained from the resulting models are consolidated according to model-averaging mechanisms to account for model uncertainty. The approach is evaluated on real data and compared to its nonrobust Bayesian counterpart, the traditional frequentist approach, and a commonly employed robust frequentist method. Detailed guidelines to automate the entire statistical procedure are provided. All required code is made available, see ArXiv:1711.06341.

To appear in Journal of Applied Statistics

References in corpus (3)

• Weak convergence of Metropolis algorithms for non-i.i.d. target distributions
• Robustness to outliers in location-scale parameter model using log-regularly varying distributions
• Weak Convergence and Optimal Tuning of the Reversible Jump Algorithm

Cited by in corpus (5)

• Informed reversible jump algorithms
• An asymptotic Peskun ordering and its application to lifted samplers
• Theoretical properties of Bayesian Student-$t$ linear regression
• Robustness against conflicting prior information in regression
• Robust heavy-tailed versions of generalized linear models with applications in actuarial science