A Shrinkage Principle for Heavy-Tailed Data: High-Dimensional Robust Low-Rank Matrix Recovery

Abstract

This paper introduces a simple principle for robust high-dimensional statistical inference via an appropriate shrinkage on the data. This widens the scope of high-dimensional techniques, reducing the moment conditions from sub-exponential or sub-Gaussian distributions to merely bounded second or fourth moment. As an illustration of this principle, we focus on robust estimation of the low-rank matrix ${\Theta }^{\ast }$ from the trace regression model $Y=Tr({\Theta }^{\ast T}X)+\epsilon$. It encompasses four popular problems: sparse linear models, compressed sensing, matrix completion and multi-task regression. We propose to apply penalized least-squares approach to appropriately truncated or shrunk data. Under only bounded $2+\delta$ moment condition on the response, the proposed robust methodology yields an estimator that possesses the same statistical error rates as previous literature with sub-Gaussian errors. For sparse linear models and multi-tasking regression, we further allow the design to have only bounded fourth moment and obtain the same statistical rates, again, by appropriate shrinkage of the design matrix. As a byproduct, we give a robust covariance matrix estimator and establish its concentration inequality in terms of the spectral norm when the random samples have only bounded fourth moment. Extensive simulations have been carried out to support our theories.

57 pages

References in corpus (4)

• Nearly unbiased variable selection under minimax concave penalty
• Restricted strong convexity and weighted matrix completion: Optimal bounds with noise
• Robust Estimation of High-Dimensional Mean Regression
• Max-Norm Optimization for Robust Matrix Recovery

Cited by in corpus (13)

• Distributed Estimation of Principal Eigenspaces
• Estimation of the covariance structure of heavy-tailed distributions
• Principal component analysis for big data
• Robust high dimensional factor models with applications to statistical machine learning
• Structured signal recovery from non-linear and heavy-tailed measurements
• On Stein's Identity and Near-Optimal Estimation in High-dimensional Index Models
• Structured Recovery with Heavy-tailed Measurements: A Thresholding Procedure and Optimal Rates
• Robust Modifications of U-statistics and Applications to Covariance Estimation Problems
• Optimal link prediction with matrix logistic regression
• Simultaneous Inference for High Dimensional Mean Vectors
• Adaptive Huber Regression on Markov-dependent Data
• II. High Dimensional Estimation under Weak Moment Assumptions: Structured Recovery and Matrix Estimation
• Robust Structured Statistical Estimation via Conditional Gradient Type Methods