Deep matrix factorizations

Abstract

Constrained low-rank matrix approximations have been known for decades as powerful linear dimensionality reduction techniques to be able to extract the information contained in large data sets in a relevant way. However, such low-rank approaches are unable to mine complex, interleaved features that underlie hierarchical semantics. Recently, deep matrix factorization (deep MF) was introduced to deal with the extraction of several layers of features and has been shown to reach outstanding performances on unsupervised tasks. Deep MF was motivated by the success of deep learning, as it is conceptually close to some neural networks paradigms. In this paper, we present the main models, algorithms, and applications of deep MF through a comprehensive literature review. We also discuss theoretical questions and perspectives of research.

References in corpus (29)

• Tensor Decomposition for Signal Processing and Machine Learning
• Nonlinear unmixing of hyperspectral images: models and algorithms
• Deep Learning: A Critical Appraisal
• Deep Variational Information Bottleneck
• Fast and Robust Recursive Algorithms for Separable Nonnegative Matrix Factorization
• The Why and How of Nonnegative Matrix Factorization
• Nonnegative Matrix Factorization for Signal and Data Analytics: Identifiability, Algorithms, and Applications
• Deep Learning of Part-based Representation of Data Using Sparse Autoencoders with Nonnegativity Constraints
• A Flexible and Efficient Algorithmic Framework for Constrained Matrix and Tensor Factorization
• Spectral Unmixing of Hyperspectral Imagery using Multilayer NMF
• Two Algorithms for Orthogonal Nonnegative Matrix Factorization with Application to Clustering
• A Practical Algorithm for Topic Modeling with Provable Guarantees
• On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
• A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks
• Sparse and Unique Nonnegative Matrix Factorization Through Data Preprocessing
• Successive Nonnegative Projection Algorithm for Robust Nonnegative Blind Source Separation
• Blind Audio Source Separation with Minimum-Volume Beta-Divergence NMF
• Generic uniqueness conditions for the canonical polyadic decomposition and INDSCAL
• The Global Landscape of Neural Networks: An Overview
• Implicit Regularization in Deep Matrix Factorization
• Accelerating Nonnegative Matrix Factorization Algorithms using Extrapolation
• Introduction to Nonnegative Matrix Factorization
• Width Provably Matters in Optimization for Deep Linear Neural Networks
• Dropout as a Low-Rank Regularizer for Matrix Factorization
• Deep Approximately Orthogonal Nonnegative Matrix Factorization for Clustering
• Near-Convex Archetypal Analysis
• End-to-end Learning of a Convolutional Neural Network via Deep Tensor Decomposition
• Learning the Hierarchical Parts of Objects by Deep Non-Smooth Nonnegative Matrix Factorization
• Deep NMF Topic Modeling

Cited by in corpus (5)

• Advanced Unsupervised Learning: A Comprehensive Overview of Multi-View Clustering Techniques
• Recommender systems based on graph embedding techniques: A comprehensive review
• A consistent and flexible framework for deep matrix factorizations
• Deep Nonnegative Matrix Factorization with Beta Divergences
• A Deep Latent Factor Graph Clustering with Fairness-Utility Trade-off Perspective