Tight convex relaxations for sparse matrix factorization

Abstract

Based on a new atomic norm, we propose a new convex formulation for sparse matrix factorization problems in which the number of nonzero elements of the factors is assumed fixed and known. The formulation counts sparse PCA with multiple factors, subspace clustering and low-rank sparse bilinear regression as potential applications. We compute slow rates and an upper bound on the statistical dimension of the suggested norm for rank 1 matrices, showing that its statistical dimension is an order of magnitude smaller than the usual $\ell \_1$-norm, trace norm and their combinations. Even though our convex formulation is in theory hard and does not lead to provably polynomial time algorithmic schemes, we propose an active set algorithm leveraging the structure of the convex problem to solve it and show promising numerical results.

References in corpus (11)

• Generalized power method for sparse principal component analysis
• A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
• Optimal Solutions for Sparse Principal Component Analysis
• Truncated Power Method for Sparse Eigenvalue Problems
• Computational and Statistical Tradeoffs via Convex Relaxation
• Estimation of Simultaneously Sparse and Low Rank Matrices
• Convex Sparse Matrix Factorizations
• Living on the edge: Phase transitions in convex programs with random data
• Convex relaxations of structured matrix factorizations
• Sharp MSE Bounds for Proximal Denoising
• Simultaneously Structured Models with Application to Sparse and Low-rank Matrices

Cited by in corpus (6)

• A convex variational model for learning convolutional image atoms from incomplete data
• A Unified Approach to Uniform Signal Recovery From Non-Linear Observations
• Optimal convex lifted sparse phase retrieval and PCA with an atomic matrix norm regularizer
• Sparsified Linear Programming for Zero-Sum Equilibrium Finding
• The Nonconvex Geometry of Linear Inverse Problems
• Approximate Frank-Wolfe Algorithms over Graph-structured Support Sets