Closed-Form, Provable, and Robust PCA via Leverage Statistics and Innovation Search

Abstract

The idea of Innovation Search, which was initially proposed for data clustering, was recently used for outlier detection. In the application of Innovation Search for outlier detection, the directions of innovation were utilized to measure the innovation of the data points. We study the Innovation Values computed by the Innovation Search algorithm under a quadratic cost function and it is proved that Innovation Values with the new cost function are equivalent to Leverage Scores. This interesting connection is utilized to establish several theoretical guarantees for a Leverage Score based robust PCA method and to design a new robust PCA method. The theoretical results include performance guarantees with different models for the distribution of outliers and the distribution of inliers. In addition, we demonstrate the robustness of the algorithms against the presence of noise. The numerical and theoretical studies indicate that while the presented approach is fast and closed-form, it can outperform most of the existing algorithms.

arXiv commentPublished in IEEE Transactions on Signal Processing. arXiv admin note: text overlap with arXiv:1912.12988

References in corpus (15)

• Rank-Sparsity Incoherence for Matrix Decomposition
• A geometric analysis of subspace clustering with outliers
• Robust PCA via Outlier Pursuit
• Optimal Algorithms for $L_1$-subspace Signal Processing
• Coherence Pursuit: Fast, Simple, and Robust Principal Component Analysis
• Two Proposals for Robust PCA using Semidefinite Programming
• Robust computation of linear models by convex relaxation
• Greedy Subspace Clustering
• Innovation Pursuit: A New Approach to Subspace Clustering
• Robust recovery of multiple subspaces by geometric l_p minimization
• Fast, Robust and Non-convex Subspace Recovery
• Principal Component Analysis with Contaminated Data: The High Dimensional Case
• Algorithms and Hardness for Robust Subspace Recovery
• Robust PCA in High-dimension: A Deterministic Approach
• Subspace Clustering via Optimal Direction Search