Parallel Algorithms for Constrained Tensor Factorization via the Alternating Direction Method of Multipliers
arXiv:1409.2383 · doi:10.1109/TSP.2015.2454476
Abstract
Tensor factorization has proven useful in a wide range of applications, from sensor array processing to communications, speech and audio signal processing, and machine learning. With few recent exceptions, all tensor factorization algorithms were originally developed for centralized, in-memory computation on a single machine; and the few that break away from this mold do not easily incorporate practically important constraints, such as nonnegativity. A new constrained tensor factorization framework is proposed in this paper, building upon the Alternating Direction method of Multipliers (ADMoM). It is shown that this simplifies computations, bypassing the need to solve constrained optimization problems in each iteration; and it naturally leads to distributed algorithms suitable for parallel implementation on regular high-performance computing (e.g., mesh) architectures. This opens the door for many emerging big data-enabled applications. The methodology is exemplified using nonnegativity as a baseline constraint, but the proposed framework can more-or-less readily incorporate many other types of constraints. Numerical experiments are very encouraging, indicating that the ADMoM-based nonnegative tensor factorization (NTF) has high potential as an alternative to state-of-the-art approaches.
Submitted to the IEEE Transactions on Signal Processing
References in corpus (2)
Cited by in corpus (26)
- Tensor Decomposition for Signal Processing and Machine Learning
- Consensus-ADMM for General Quadratically Constrained Quadratic Programming
- Global Convergence of ADMM in Nonconvex Nonsmooth Optimization
- A Flexible and Efficient Algorithmic Framework for Constrained Matrix and Tensor Factorization
- Parallel Algorithms for Constrained Tensor Factorization via the Alternating Direction Method of Multipliers
- Tensor Analysis and Fusion of Multimodal Brain Images
- Convergence of Bregman alternating direction method with multipliers for nonconvex composite problems
- Decomposing Linearly Constrained Nonconvex Problems by a Proximal Primal Dual Approach: Algorithms, Convergence, and Applications
- A Distributed, Asynchronous and Incremental Algorithm for Nonconvex Optimization: An ADMM Based Approach
- A Flexible Optimization Framework for Regularized Matrix-Tensor Factorizations with Linear Couplings
- Convergence of multi-block Bregman ADMM for nonconvex composite problems
- An Empirical Study of ADMM for Nonconvex Problems
- A General System for Heuristic Solution of Convex Problems over Nonconvex Sets
- Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems
- Coupled Graphs and Tensor Factorization for Recommender Systems and Community Detection
- The Application of Preconditioned Alternating Direction Method of Multipliers in Depth from Focal Stack
- Iteration-complexity of a Jacobi-type non-Euclidean ADMM for multi-block linearly constrained nonconvex programs
- Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization
- Stochastic Variance-Reduced ADMM
- Alternating minimization algorithms for graph regularized tensor completion
- Joint Embedding of Meta-Path and Meta-Graph for Heterogeneous Information Networks
- Sparse Nonnegative CANDECOMP/PARAFAC Decomposition in Block Coordinate Descent Framework: A Comparison Study
- Simple and Efficient Parallelization for Probabilistic Temporal Tensor Factorization
- Non-negative Factorization of the Occurrence Tensor from Financial Contracts
- Convex Optimization For Non-Convex Problems via Column Generation
- Broad Learning for Healthcare