Robust Parametric Classification and Variable Selection by a Minimum Distance Criterion

Abstract

We investigate a robust penalized logistic regression algorithm based on a minimum distance criterion. Influential outliers are often associated with the explosion of parameter vector estimates, but in the context of standard logistic regression, the bias due to outliers always causes the parameter vector to implode, that is shrink towards the zero vector. Thus, using LASSO-like penalties to perform variable selection in the presence of outliers can result in missed detections of relevant covariates. We show that by choosing a minimum distance criterion together with an Elastic Net penalty, we can simultaneously find a parsimonious model and avoid estimation implosion even in the presence of many outliers in the important small $n$ large $p$ situation. Implementation using an MM algorithm is described and performance evaluated.

41 pages, 9 figures

References in corpus (17)

• Discussion of "Least angle regression" by Efron et al
• Rejoinder to "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Pathwise coordinate optimization
• Coordinate descent algorithms for lasso penalized regression
• Piecewise linear regularized solution paths
• High-dimensional generalized linear models and the lasso
• Sparse least trimmed squares regression for analyzing high-dimensional large data sets
• Sparse logistic principal components analysis for binary data
• MM Algorithms for Minimizing Nonsmoothly Penalized Objective Functions

Cited by in corpus (8)

• Wavelet-domain regression and predictive inference in psychiatric neuroimaging
• Robust and Sparse Regression in GLM by Stochastic Optimization
• On Difference Between Two Types of $\gamma$-divergence for Regression
• Testing equivalence to binary generalized linear models with application to logistic regression
• MM for Penalized Estimation
• Penalized robust estimators in logistic regression with applications to sparse models
• Robust Estimation of Sparse Precision Matrix using Adaptive Weighted Graphical Lasso Approach
• Robust Analysis for Resilient AI System