Stochastic Gradients for Large-Scale Tensor Decomposition

Abstract

Tensor decomposition is a well-known tool for multiway data analysis. This work proposes using stochastic gradients for efficient generalized canonical polyadic (GCP) tensor decomposition of large-scale tensors. GCP tensor decomposition is a recently proposed version of tensor decomposition that allows for a variety of loss functions such as Bernoulli loss for binary data or Huber loss for robust estimation. The stochastic gradient is formed from randomly sampled elements of the tensor and is efficient because it can be computed using the sparse matricized-tensor-times-Khatri-Rao product (MTTKRP) tensor kernel. For dense tensors, we simply use uniform sampling. For sparse tensors, we propose two types of stratified sampling that give precedence to sampling nonzeros. Numerical results demonstrate the advantages of the proposed approach and its scalability to large-scale problems.

References in corpus (10)

• Scalable Tensor Factorizations for Incomplete Data
• Correcting systematic effects in a large set of photometric lightcurves
• Sketching as a Tool for Numerical Linear Algebra
• On Tensors, Sparsity, and Nonnegative Factorizations
• Subspace Learning and Imputation for Streaming Big Data Matrices and Tensors
• A Practical Randomized CP Tensor Decomposition
• A Scalable Generative Graph Model with Community Structure
• Generalized Canonical Polyadic Tensor Decomposition
• Newton-Based Optimization for Kullback-Leibler Nonnegative Tensor Factorizations
• Software for Sparse Tensor Decomposition on Emerging Computing Architectures

Cited by in corpus (6)

• Grassmannian Optimization for Online Tensor Completion and Tracking with the t-SVD
• An AO-ADMM approach to constraining PARAFAC2 on all modes
• Stochastic Mirror Descent for Low-Rank Tensor Decomposition Under Non-Euclidean Losses
• Performance of the low-rank tensor-train SVD (TT-SVD) for large dense tensors on modern multi-core CPUs
• Distributed-Memory Randomized Algorithms for Sparse Tensor CP Decomposition
• Zero-Truncated Poisson Regression for Sparse Multiway Count Data Corrupted by False Zeros