Structured Sparse Principal Component Analysis

Abstract

We present an extension of sparse PCA, or sparse dictionary learning, where the sparsity patterns of all dictionary elements are structured and constrained to belong to a prespecified set of shapes. This \emph{structured sparse PCA} is based on a structured regularization recently introduced by [1]. While classical sparse priors only deal with \textit{cardinality}, the regularization we use encodes higher-order information about the data. We propose an efficient and simple optimization procedure to solve this problem. Experiments with two practical tasks, face recognition and the study of the dynamics of a protein complex, demonstrate the benefits of the proposed structured approach over unstructured approaches.

References in corpus (3)

• Consistency of the group Lasso and multiple kernel learning
• Optimal Solutions for Sparse Principal Component Analysis
• Convex Sparse Matrix Factorizations

Cited by in corpus (38)

• Structured Variable Selection with Sparsity-Inducing Norms
• Proximal Methods for Hierarchical Sparse Coding
• Optimal detection of sparse principal components in high dimension
• OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage
• Network Flow Algorithms for Structured Sparsity
• Fast Robust PCA on Graphs
• Stochastic Majorization-Minimization Algorithms for Large-Scale Optimization
• Convex and Network Flow Optimization for Structured Sparsity
• Bayesian Group Factor Analysis
• Structured sparsity-inducing norms through submodular functions
• Sum-of-Squares Lower Bounds for Sparse PCA
• CUR from a Sparse Optimization Viewpoint
• Sparse and Functional Principal Components Analysis
• A Convex Formulation for Learning Scale-Free Networks via Submodular Relaxation
• Bayesian Variable Selection for Globally Sparse Probabilistic PCA
• Sparse PCA via Bipartite Matchings
• Principal Sensitivity Analysis
• Collaborative Filtering via Group-Structured Dictionary Learning
• Bayesian group latent factor analysis with structured sparsity
• Half-Space Proximal Stochastic Gradient Method for Group-Sparsity Regularized Problem
• Optimal Projected Variance Group-Sparse Block PCA
• A Fast Data Driven Shrinkage of Singular Values for Arbitrary Rank Signal Matrix Denoising
• Sparse principal component analysis and its $l_1$-relaxation
• The Flip Side of the Reweighted Coin: Duality of Adaptive Dropout and Regularization
• Dynamic Tensor Clustering
• A sparse decomposition of low rank symmetric positive semi-definite matrices
• Projection Algorithms for Non-Convex Minimization with Application to Sparse Principal Component Analysis
• A Method for Finding Structured Sparse Solutions to Non-negative Least Squares Problems with Applications
• Coarse-to-Fine Salient Object Detection with Low-Rank Matrix Recovery
• High-Dimensional Uncertainty Quantification via Active and Rank-Adaptive Tensor Regression
• High-Dimensional Uncertainty Quantification via Tensor Regression with Rank Determination and Adaptive Sampling
• Analysis of Truncated Orthogonal Iteration for Sparse Eigenvector Problems
• Incorporating prior knowledge about structural constraints in model identification
• Structured Sparse Aggregation
• Low-Rank Reorganization via Proportional Hazards Non-negative Matrix Factorization Unveils Survival Associated Gene Clusters
• Information Projection and Approximate Inference for Structured Sparse Variables
• Stay on path: PCA along graph paths
• Order-preserving factor analysis (OPFA)