The Singular Value Decomposition, Applications and Beyond

Abstract

The singular value decomposition (SVD) is not only a classical theory in matrix computation and analysis, but also is a powerful tool in machine learning and modern data analysis. In this tutorial we first study the basic notion of SVD and then show the central role of SVD in matrices. Using majorization theory, we consider variational principles of singular values and eigenvalues. Built on SVD and a theory of symmetric gauge functions, we discuss unitarily invariant norms, which are then used to formulate general results for matrix low rank approximation. We study the subdifferentials of unitarily invariant norms. These results would be potentially useful in many machine learning problems such as matrix completion and matrix data classification. Finally, we discuss matrix low rank approximation and its recent developments such as randomized SVD, approximate matrix multiplication, CUR decomposition, and Nystrom approximation. Randomized algorithms are important approaches to large scale SVD as well as fast matrix computations.

References in corpus (5)

• A kernel method for canonical correlation analysis
• Optimal approximate matrix product in terms of stable rank
• CUR from a Sparse Optimization Viewpoint
• Improved Analyses of the Randomized Power Method and Block Lanczos Method
• Using a Non-Commutative Bernstein Bound to Approximate Some Matrix Algorithms in the Spectral Norm

Cited by in corpus (4)

• Variational Quantum Singular Value Decomposition
• Scalable Kernel K-Means Clustering with Nystrom Approximation: Relative-Error Bounds
• Integrating multiple random sketches for singular value decomposition
• Nonconvex One-bit Single-label Multi-label Learning