Low Rank Regularization: A Review

Abstract

Low rank regularization, in essence, involves introducing a low rank or approximately low rank assumption for matrix we aim to learn, which has achieved great success in many fields including machine learning, data mining and computer version. Over the last decade, much progress has been made in theories and practical applications. Nevertheless, the intersection between them is very slight. In order to construct a bridge between practical applications and theoretical research, in this paper we provide a comprehensive survey for low rank regularization. We first review several traditional machine learning models using low rank regularization, and then show their (or their variants) applications in solving practical issues, such as non-rigid structure from motion and image denoising. Subsequently, we summarize the regularizers and optimization methods that achieve great success in traditional machine learning tasks but are rarely seen in solving practical issues. Finally, we provide a discussion and comparison for some representative regularizers including convex and non-convex relaxations. Extensive experimental results demonstrate that non-convex regularizers can provide a large advantage over the nuclear norm, the regularizer widely used in solving practical issues.

arXiv comment16 pages,4 figures,4 tables

References in corpus (19)

• Nearly unbiased variable selection under minimax concave penalty
• Robust Recovery of Subspace Structures by Low-Rank Representation
• The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices
• Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview
• Non-convex Optimization for Machine Learning
• Generalized Nonconvex Nonsmooth Low-Rank Minimization
• Discriminative Block-Diagonal Representation Learning for Image Recognition
• Iterative Views Agreement: An Iterative Low-Rank based Structured Optimization Method to Multi-View Spectral Clustering
• Clustering Partially Observed Graphs via Convex Optimization
• Robust Subspace Clustering with Compressed Data
• Matrix Completion with Noisy Entries and Outliers
• Scalable Algorithms for Tractable Schatten Quasi-Norm Minimization
• Harmonic Mean Iteratively Reweighted Least Squares for Low-Rank Matrix Recovery
• Trained Rank Pruning for Efficient Deep Neural Networks
• Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective
• Fast and Accurate Tensor Completion with Total Variation Regularized Tensor Trains
• Alternating Iteratively Reweighted Minimization Algorithms for Low-Rank Matrix Factorization
• Efficient Inexact Proximal Gradient Algorithm for Nonconvex Problems
• FasTer: Fast Tensor Completion with Nonconvex Regularization