On Tensors, Sparsity, and Nonnegative Factorizations

Abstract

Tensors have found application in a variety of fields, ranging from chemometrics to signal processing and beyond. In this paper, we consider the problem of multilinear modeling of sparse count data. Our goal is to develop a descriptive tensor factorization model of such data, along with appropriate algorithms and theory. To do so, we propose that the random variation is best described via a Poisson distribution, which better describes the zeros observed in the data as compared to the typical assumption of a Gaussian distribution. Under a Poisson assumption, we fit a model to observed data using the negative log-likelihood score. We present a new algorithm for Poisson tensor factorization called CANDECOMP-PARAFAC Alternating Poisson Regression (CP-APR) that is based on a majorization-minimization approach. It can be shown that CP-APR is a generalization of the Lee-Seung multiplicative updates. We show how to prevent the algorithm from converging to non-KKT points and prove convergence of CP-APR under mild conditions. We also explain how to implement CP-APR for large-scale sparse tensors and present results on several data sets, both real and simulated.

Cited by in corpus (65)

• Deep Representation Learning of Patient Data from Electronic Health Records (EHR): A Systematic Review
• Community detection, link prediction, and layer interdependence in multilayer networks
• Two Algorithms for Orthogonal Nonnegative Matrix Factorization with Application to Clustering
• Generalized Canonical Polyadic Tensor Decomposition
• Rank regularization and Bayesian inference for tensor completion and extrapolation
• STORE: Sparse Tensor Response Regression and Neuroimaging Analysis
• Deep Coevolutionary Network: Embedding User and Item Features for Recommendation
• Spatiotemporal Tensor Completion for Improved Urban Traffic Imputation
• Stochastic Gradients for Large-Scale Tensor Decomposition
• Algorithms for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence
• Newton-Based Optimization for Kullback-Leibler Nonnegative Tensor Factorizations
• Bayesian Poisson Tucker Decomposition for Learning the Structure of International Relations
• Privacy-Preserving Tensor Factorization for Collaborative Health Data Analysis
• On the Complexity of Robust PCA and ${\ell }_1$-norm Low-Rank Matrix Approximation
• A Flexible Optimization Framework for Regularized Matrix-Tensor Factorizations with Linear Couplings
• Zero-Truncated Poisson Tensor Factorization for Massive Binary Tensors
• A literature survey of matrix methods for data science
• Regularized Tensor Factorizations and Higher-Order Principal Components Analysis
• A universality theorem for nonnegative matrix factorizations
• Low-Rank Matrix Approximation in the Infinity Norm
• Clustering Boolean Tensors
• Stochastic Mirror Descent for Low-Rank Tensor Decomposition Under Non-Euclidean Losses
• Boltzmann machines as two-dimensional tensor networks
• SPARTan: Scalable PARAFAC2 for Large & Sparse Data
• General Tensor Spectral Co-clustering for Higher-Order Data
• Legendre Decomposition for Tensors
• Accelerating Block Coordinate Descent for Nonnegative Tensor Factorization
• Scalable Bayesian Non-Negative Tensor Factorization for Massive Count Data
• Phenotyping using Structured Collective Matrix Factorization of Multi--source EHR Data
• Multiplicative Updates for NMF with $\beta$-Divergences under Disjoint Equality Constraints
• Learning Inter-Modal Correspondence and Phenotypes from Multi-Modal Electronic Health Records
• Bayesian Poisson Tensor Factorization for Inferring Multilateral Relations from Sparse Dyadic Event Counts
• Unsupervised EHR-based Phenotyping via Matrix and Tensor Decompositions
• Shape Constrained Tensor Decompositions using Sparse Representations in Over-Complete Libraries
• Supervised tensor decomposition with features on multiple modes
• A generalizable framework for low-rank tensor completion with numerical priors
• Generating coupled cluster code for modern distributed memory tensor software
• Regularized and Smooth Double Core Tensor Factorization for Heterogeneous Data
• Multiresolution Tensor Decomposition for Multiple Spatial Passing Networks
• Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization
• Variational Auto-encoder Based Bayesian Poisson Tensor Factorization for Sparse and Imbalanced Count Data
• Taming numerical imprecision by adapting the KL divergence to negative probabilities
• tHoops: A Multi-Aspect Analytical Framework Spatio-Temporal Basketball Data
• Search Engine Similarity Analysis: A Combined Content and Rankings Approach
• Scalable Boolean Tensor Factorizations using Random Walks
• Link Prediction Under Imperfect Detection: Collaborative Filtering for Ecological Networks
• Provable Sparse Tensor Decomposition
• Several Approximation Algorithms for Sparse Best Rank-1 Approximation to Higher-Order Tensors
• Tensor Decomposition via Variational Auto-Encoder
• Nonnegative rank depends on the field II
• Nesterov Acceleration of Alternating Least Squares for Canonical Tensor Decomposition: Momentum Step Size Selection and Restart Mechanisms
• TULIP: A Toolbox for Linear Discriminant Analysis with Penalties
• Sparse Tensor Additive Regression
• PASTA: A Parallel Sparse Tensor Algorithm Benchmark Suite
• Zero-Truncated Poisson Regression for Sparse Multiway Count Data Corrupted by False Zeros
• A Doubly-Enhanced EM Algorithm for Model-Based Tensor Clustering
• A quadratically convergent proximal algorithm for nonnegative tensor decomposition
• Tensor Decompositions for Count Data that Leverage Stochastic and Deterministic Optimization
• A Sparse Tensor Generator with Efficient Feature Extraction
• Poly-NL: Linear Complexity Non-local Layers with Polynomials
• Integrating Hypertension Phenotype and Genotype with Hybrid Non-negative Matrix Factorization
• Covariate-Adjusted Tensor Classification in High-Dimensions
• Low-Rank Approximation of Weighted Tree Automata
• Cyclic Coordinate Update Algorithms for Fixed-Point Problems: Analysis and Applications
• Fast Tucker Rank Reduction for Non-Negative Tensors Using Mean-Field Approximation