A Flexible and Efficient Algorithmic Framework for Constrained Matrix and Tensor Factorization
arXiv:1506.04209 · doi:10.1109/TSP.2016.2576427
Abstract
We propose a general algorithmic framework for constrained matrix and tensor factorization, which is widely used in signal processing and machine learning. The new framework is a hybrid between alternating optimization (AO) and the alternating direction method of multipliers (ADMM): each matrix factor is updated in turn, using ADMM, hence the name AO-ADMM. This combination can naturally accommodate a great variety of constraints on the factor matrices, and almost all possible loss measures for the fitting. Computation caching and warm start strategies are used to ensure that each update is evaluated efficiently, while the outer AO framework exploits recent developments in block coordinate descent (BCD)-type methods which help ensure that every limit point is a stationary point, as well as faster and more robust convergence in practice. Three special cases are studied in detail: non-negative matrix/tensor factorization, constrained matrix/tensor completion, and dictionary learning. Extensive simulations and experiments with real data are used to showcase the effectiveness and broad applicability of the proposed framework.
References in corpus (2)
Cited by in corpus (35)
- Tensor Decomposition for Signal Processing and Machine Learning
- Nonnegative Matrix Factorization for Signal and Data Analytics: Identifiability, Algorithms, and Applications
- Dynamical Variational Autoencoders: A Comprehensive Review
- Generalized Canonical Polyadic Tensor Decomposition
- Deep matrix factorizations
- Tensor-Based Channel Estimation for Dual-Polarized Massive MIMO Systems
- Learning From Hidden Traits: Joint Factor Analysis and Latent Clustering
- Algorithms for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence
- A Nonconvex Splitting Method for Symmetric Nonnegative Matrix Factorization: Convergence Analysis and Optimality
- Tensors, Learning, and 'Kolmogorov Extension' for Finite-alphabet Random Vectors
- Computing Large-Scale Matrix and Tensor Decomposition with Structured Factors: A Unified Nonconvex Optimization Perspective
- Blind Multiclass Ensemble Classification
- An AO-ADMM approach to constraining PARAFAC2 on all modes
- A Flexible Optimization Framework for Regularized Matrix-Tensor Factorizations with Linear Couplings
- Identification of Overlapping Communities via Constrained Egonet Tensor Decomposition
- Stochastic Mirror Descent for Low-Rank Tensor Decomposition Under Non-Euclidean Losses
- Accelerating Block Coordinate Descent for Nonnegative Tensor Factorization
- Provable Online CP/PARAFAC Decomposition of a Structured Tensor via Dictionary Learning
- PARAFAC2-based Coupled Matrix and Tensor Factorizations
- PARAFAC2 AO-ADMM: Constraints in all modes
- MultiHU-TD: Multifeature Hyperspectral Unmixing Based on Tensor Decomposition
- Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization
- Leveraging Two Reference Functions in Block Bregman Proximal Gradient Descent for Non-convex and Non-Lipschitz Problems
- PLANC: Parallel Low Rank Approximation with Non-negativity Constraints
- COPA: Constrained PARAFAC2 for Sparse & Large Datasets
- Efficient Constrained Tensor Factorization by Alternating Optimization with Primal-Dual Splitting
- Inertial Block Proximal Methods for Non-Convex Non-Smooth Optimization
- SWoTTeD: An Extension of Tensor Decomposition to Temporal Phenotyping
- tPARAFAC2: Tracking evolving patterns in (incomplete) temporal data
- A Time-aware tensor decomposition for tracking evolving patterns
- Efficient Multidimensional Functional Data Analysis Using Marginal Product Basis Systems
- Hyperspectral Super-resolution: A Coupled Nonnegative Block-term Tensor Decomposition Approach
- PARAFAC2-based Coupled Matrix and Tensor Factorizations with Constraints
- A quadratically convergent proximal algorithm for nonnegative tensor decomposition
- Convex Optimization For Non-Convex Problems via Column Generation