Robust PCA as Bilinear Decomposition with Outlier-Sparsity Regularization

Abstract

Principal component analysis (PCA) is widely used for dimensionality reduction, with well-documented merits in various applications involving high-dimensional data, including computer vision, preference measurement, and bioinformatics. In this context, the fresh look advocated here permeates benefits from variable selection and compressive sampling, to robustify PCA against outliers. A least-trimmed squares estimator of a low-rank bilinear factor analysis model is shown closely related to that obtained from an ${\ell }_0$-(pseudo)norm-regularized criterion encouraging sparsity in a matrix explicitly modeling the outliers. This connection suggests robust PCA schemes based on convex relaxation, which lead naturally to a family of robust estimators encompassing Huber's optimal M-class as a special case. Outliers are identified by tuning a regularization parameter, which amounts to controlling sparsity of the outlier matrix along the whole robustification path of (group) least-absolute shrinkage and selection operator (Lasso) solutions. Beyond its neat ties to robust statistics, the developed outlier-aware PCA framework is versatile to accommodate novel and scalable algorithms to: i) track the low-rank signal subspace robustly, as new data are acquired in real time; and ii) determine principal components robustly in (possibly) infinite-dimensional feature spaces. Synthetic and real data tests corroborate the effectiveness of the proposed robust PCA schemes, when used to identify aberrant responses in personality assessment surveys, as well as unveil communities in social networks, and intruders from video surveillance data.

30 pages, submitted to IEEE Transactions on Signal Processing

References in corpus (4)

• Robust PCA via Outlier Pursuit
• Online Robust Subspace Tracking from Partial Information
• In-network Sparsity-regularized Rank Minimization: Algorithms and Applications
• From Sparse Signals to Sparse Residuals for Robust Sensing

Cited by in corpus (14)

• Robust Subspace Learning: Robust PCA, Robust Subspace Tracking, and Robust Subspace Recovery
• Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset
• An Online Algorithm for Separating Sparse and Low-dimensional Signal Sequences from their Sum
• Dynamic Anomalography: Tracking Network Anomalies via Sparsity and Low Rank
• L1-norm Principal-Component Analysis of Complex Data
• Randomized Robust Subspace Recovery for High Dimensional Data Matrices
• Low-Rank Positive Semidefinite Matrix Recovery from Corrupted Rank-One Measurements
• Learning efficient sparse and low rank models
• Weighted Low-Rank Approximation of Matrices and Background Modeling
• Online (and Offline) Robust PCA: Novel Algorithms and Performance Guarantees
• Learning Robust Low-Rank Representations
• A New Approach to Dimensionality Reduction for Anomaly Detection in Data Traffic
• Atomic Norm Denoising-Based Joint Channel Estimation and Faulty Antenna Detection for Massive MIMO
• DPCA: Dimensionality Reduction for Discriminative Analytics of Multiple Large-Scale Datasets