A Survey on Nonconvex Regularization Based Sparse and Low-Rank Recovery in Signal Processing, Statistics, and Machine Learning

Abstract

In the past decade, sparse and low-rank recovery have drawn much attention in many areas such as signal/image processing, statistics, bioinformatics and machine learning. To achieve sparsity and/or low-rankness inducing, the ${\ell }_1$ norm and nuclear norm are of the most popular regularization penalties due to their convexity. While the ${\ell }_1$ and nuclear norm are convenient as the related convex optimization problems are usually tractable, it has been shown in many applications that a nonconvex penalty can yield significantly better performance. In recent, nonconvex regularization based sparse and low-rank recovery is of considerable interest and it in fact is a main driver of the recent progress in nonconvex and nonsmooth optimization. This paper gives an overview of this topic in various fields in signal processing, statistics and machine learning, including compressive sensing (CS), sparse regression and variable selection, sparse signals separation, sparse principal component analysis (PCA), large covariance and inverse covariance matrices estimation, matrix completion, and robust PCA. We present recent developments of nonconvex regularization based sparse and low-rank recovery in these fields, addressing the issues of penalty selection, applications and the convergence of nonconvex algorithms. Code is available at https://github.com/FWen/ncreg.git.

arXiv comment22 pages

References in corpus (26)

• Nearly unbiased variable selection under minimax concave penalty
• Message Passing Algorithms for Compressed Sensing
• Challenges of Big Data Analysis
• A fast approach for overcomplete sparse decomposition based on smoothed L0 norm
• High-Resolution Radar via Compressed Sensing
• Nonconcave penalized likelihood with a diverging number of parameters
• Generalized power method for sparse principal component analysis
• Sparse Regularization via Convex Analysis
• Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview
• Guaranteed Matrix Completion via Non-convex Factorization
• Optimal Solutions for Sparse Principal Component Analysis
• $L_{1/2}$ Regularization: Convergence of Iterative Half Thresholding Algorithm
• Simultaneously sparse and low-rank abundance matrix estimation for hyperspectral image unmixing
• Non-convex Robust PCA
• Enhanced Low-Rank Matrix Approximation
• Convergence of Bregman alternating direction method with multipliers for nonconvex composite problems
• Convexity in source separation: Models, geometry, and algorithms
• L0 Sparse Inverse Covariance Estimation
• Global solutions to folded concave penalized nonconvex learning
• Harnessing Structures in Big Data via Guaranteed Low-Rank Matrix Estimation
• Does ${\ell }_p$-minimization outperform ${\ell }_1$-minimization?
• The Trimmed Lasso: Sparsity and Robustness
• Schatten-$p$ Quasi-Norm Regularized Matrix Optimization via Iterative Reweighted Singular Value Minimization
• Unified Scalable Equivalent Formulations for Schatten Quasi-Norms
• Deterministic Approximate Methods for Maximum Consensus Robust Fitting
• Efficient Outlier Removal in Large Scale Global Structure-from-Motion

Cited by in corpus (1)

• The coupling effect of Lipschitz regularization in deep neural networks