Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization
arXiv:1901.05529
Abstract
This work considers the problem of computing the canonical polyadic decomposition (CPD) of large tensors. Prior works mostly leverage data sparsity to handle this problem, which is not suitable for handling dense tensors that often arise in applications such as medical imaging, computer vision, and remote sensing. Stochastic optimization is known for its low memory cost and per-iteration complexity when handling dense data. However, exisiting stochastic CPD algorithms are not flexible enough to incorporate a variety of constraints/regularizations that are of interest in signal and data analytics. Convergence properties of many such algorithms are also unclear. In this work, we propose a stochastic optimization framework for large-scale CPD with constraints/regularizations. The framework works under a doubly randomized fashion, and can be regarded as a judicious combination of randomized block coordinate descent (BCD) and stochastic proximal gradient (SPG). The algorithm enjoys lightweight updates and small memory footprint. In addition, this framework entails considerable flexibility---many frequently used regularizers and constraints can be readily handled under the proposed scheme. The approach is also supported by convergence analysis. Numerical results on large-scale dense tensors are employed to showcase the effectiveness of the proposed approach.
arXiv commentaccepted version; IEEE Transactions on Signal Processing
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- ADADELTA: An Adaptive Learning Rate Method
- Tensor Decomposition for Signal Processing and Machine Learning
- On Tensors, Sparsity, and Nonnegative Factorizations
- Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach
- A Flexible and Efficient Algorithmic Framework for Constrained Matrix and Tensor Factorization
- Parallel Algorithms for Constrained Tensor Factorization via the Alternating Direction Method of Multipliers
- On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization
- Joint Tensor Factorization and Outlying Slab Suppression with Applications
- A Method of Moments for Mixture Models and Hidden Markov Models
- Tensors, Learning, and 'Kolmogorov Extension' for Finite-alphabet Random Vectors
- Blind Multiclass Ensemble Classification
- Randomized Block Coordinate Descent for Online and Stochastic Optimization
- Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm
- Nesterov Acceleration of Alternating Least Squares for Canonical Tensor Decomposition: Momentum Step Size Selection and Restart Mechanisms