Tensor Decomposition for Signal Processing and Machine Learning

Abstract

Tensors or {\em multi-way arrays} are functions of three or more indices $(i,j,k,\cdots )$ -- similar to matrices (two-way arrays), which are functions of two indices $(r,c)$ for (row,column). Tensors have a rich history, stretching over almost a century, and touching upon numerous disciplines; but they have only recently become ubiquitous in signal and data analytics at the confluence of signal processing, statistics, data mining and machine learning. This overview article aims to provide a good starting point for researchers and practitioners interested in learning about and working with tensors. As such, it focuses on fundamentals and motivation (using various application examples), aiming to strike an appropriate balance of breadth {\em and depth} that will enable someone having taken first graduate courses in matrix algebra and probability to get started doing research and/or developing tensor algorithms and software. Some background in applied optimization is useful but not strictly required. The material covered includes tensor rank and rank decomposition; basic tensor factorization models and their relationships and properties (including fairly good coverage of identifiability); broad coverage of algorithms ranging from alternating optimization to stochastic gradient; statistical performance analysis; and applications ranging from source separation to collaborative filtering, mixture and topic modeling, classification, and multilinear subspace learning.

revised version, overview article

References in corpus (1)

• DFacTo: Distributed Factorization of Tensors

Cited by in corpus (222)

• Channel Estimation for RIS-Aided Multiuser Millimeter-Wave Systems
• Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach
• Channel Estimation for Intelligent Reflecting Surface Assisted MIMO Systems: A Tensor Modeling Approach
• Machine-Learning-Based Diagnostics of EEG Pathology
• TensorLy: Tensor Learning in Python
• Evaluation of the MACE Force Field Architecture: from Medicinal Chemistry to Materials Science
• Tensor Methods in Computer Vision and Deep Learning
• Lecture Notes of Tensor Network Contractions
• Aesthetic-based Clothing Recommendation
• Signal Processing on Higher-Order Networks: Livin' on the Edge ... and Beyond
• Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling
• Deep Plug-and-play Prior for Low-rank Tensor Completion
• Deep matrix factorizations
• Spectrum Cartography via Coupled Block-Term Tensor Decomposition
• Particularities and commonalities of singular spectrum analysis as a method of time series analysis and signal processing
• Deep Polynomial Neural Networks
• Tensor Regression Networks
• Fast Noise Removal in Hyperspectral Images via Representative Coefficient Total Variation
• Tensor Decompositions for Identifying Directed Graph Topologies and Tracking Dynamic Networks
• Tensor-Based Channel Estimation for Dual-Polarized Massive MIMO Systems
• Tensor-Structured Coupled Cluster Theory
• Hurricane Forecasting: A Novel Multimodal Machine Learning Framework
• Coupled Tensor Decomposition for Hyperspectral and Multispectral Image Fusion with Inter-Image Variability
• DOA Estimation for Transmit Beamspace MIMO Radar via Tensor Decomposition with Vandermonde Factor Matrix
• Hyperspectral Denoising Using Unsupervised Disentangled Spatio-Spectral Deep Priors
• Provable Tensor Ring Completion
• Tensor Completion from Regular Sub-Nyquist Samples
• Double Coupled Canonical Polyadic Decomposition for Joint Blind Source Separation
• Joint DOD and DOA Estimation in Slow-Time MIMO Radar via PARAFAC Decomposition
• The condition number of join decompositions
• Tensors, Learning, and 'Kolmogorov Extension' for Finite-alphabet Random Vectors
• Block-Term Tensor Decomposition: Model Selection and Computation
• Block-Term Tensor Decomposition Model Selection and Computation: The Bayesian Way
• TenDSuR: Tensor-Based 4D Sub-Nyquist Radar
• Computing Large-Scale Matrix and Tensor Decomposition with Structured Factors: A Unified Nonconvex Optimization Perspective
• ALTO: Adaptive Linearized Storage of Sparse Tensors
• A Riemannian Trust Region Method for the Canonical Tensor Rank Approximation Problem
• Algebraic Channel Estimation Algorithms for FDD Massive MIMO systems
• A General Destriping Framework for Remote Sensing Images Using Flatness Constraint
• Blind Direction-of-Arrival Estimation in Acoustic Vector-Sensor Arrays via Tensor Decomposition and Kullback-Leibler Divergence Covariance Fitting
• Tensor-based Basis Function Learning for Three-dimensional Sound Speed Fields
• Blind Multiclass Ensemble Classification
• Tensor Regression Using Low-rank and Sparse Tucker Decompositions
• Tensor Decomposition for Model Reduction in Neural Networks: A Review
• Towards Overfitting Avoidance: Tuning-free Tensor-aided Multi-user Channel Estimation for 3D Massive MIMO Communications
• Spectral Learning on Matrices and Tensors
• From Symmetry to Geometry: Tractable Nonconvex Problems
• Decomposition of Higher-Order Spectra for Blind Multiple-Input Deconvolution, Pattern Identification and Separation
• A Low-rank Tensor Regularization Strategy for Hyperspectral Unmixing
• The Exact Solution to Rank-1 L1-norm TUCKER2 Decomposition
• Tensor Completion Algorithms in Big Data Analytics
• Pencil-based algorithms for tensor rank decomposition are not stable
• A compressive channel estimation technique robust to synchronization impairments
• A Tensor-based Approach to Joint Channel Estimation / Data Detection in Flexible Multicarrier MIMO Systems
• Interpolatory tensorial reduced order models for parametric dynamical systems
• High-dimensional rank-one nonsymmetric matrix decomposition: the spherical case
• Learning Mixtures of Separable Dictionaries for Tensor Data: Analysis and Algorithms
• Higher-Order Networks Representation and Learning: A Survey
• Stochastic Mirror Descent for Low-Rank Tensor Decomposition Under Non-Euclidean Losses
• Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Estimation from Incomplete Measurements
• Recovering Joint Probability of Discrete Random Variables from Pairwise Marginals
• SPARTan: Scalable PARAFAC2 for Large & Sparse Data
• Coupled Graphs and Tensor Factorization for Recommender Systems and Community Detection
• Efficient Nonnegative Tensor Factorization via Saturating Coordinate Descent
• Low-rank nonnegative tensor approximation via alternating projections and sketching
• Statistical mechanics of low-rank tensor decomposition
• Mode-wise Tensor Decompositions: Multi-dimensional Generalizations of CUR Decompositions
• calculus: High Dimensional Numerical and Symbolic Calculus in R
• Accelerating Block Coordinate Descent for Nonnegative Tensor Factorization
• Massive MIMO-OFDM Channel Acquisition with Time-Frequency Phase-Shifted Pilots
• Low-rank Characteristic Tensor Density Estimation Part I: Foundations
• Improved Hyperspectral Unmixing With Endmember Variability Parametrized Using an Interpolated Scaling Tensor
• CausalX: Causal Explanations and Block Multilinear Factor Analysis
• Compositional Hierarchical Tensor Factorization: Representing Hierarchical Intrinsic and Extrinsic Causal Factors
• Robust Factorization of Real-world Tensor Streams with Patterns, Missing Values, and Outliers
• A High-Performance Sparse Tensor Algebra Compiler in Multi-Level IR
• Information-theoretic Feature Selection via Tensor Decomposition and Submodularity
• PolyGAN: High-Order Polynomial Generators
• A Sharp Blockwise Tensor Perturbation Bound for Orthogonal Iteration
• Weighted Moore-Penrose inverses of arbitrary-order tensors
• Low-rank Characteristic Tensor Density Estimation Part II: Compression and Latent Density Estimation
• Multiresolution Tensor Learning for Efficient and Interpretable Spatial Analysis
• Exploring AI in Fashion: A Review of Aesthetics, Personalization, Virtual Try-On, and Forecasting
• Deep Hyperspectral and Multispectral Image Fusion with Inter-image Variability
• A generalization of Kruskal's theorem on tensor decomposition
• Provable Online CP/PARAFAC Decomposition of a Structured Tensor via Dictionary Learning
• A case study of spatiotemporal forecasting techniques for weather forecasting
• Jacobi-type algorithm for low rank orthogonal approximation of symmetric tensors and its convergence analysis
• End-to-end Learning of a Convolutional Neural Network via Deep Tensor Decomposition
• Hyperspectral Super-Resolution via Coupled Tensor Ring Factorization
• Power Iteration for Tensor PCA
• Radar Code Design for the Joint Optimization of Detection Performance and Measurement Accuracy in Track Maintenance
• F-IVM: Analytics over Relational Databases under Updates
• Average-Case Complexity of Tensor Decomposition for Low-Degree Polynomials
• Low-rank Tensor Grid for Image Completion
• Nonlocal Low-Rank Tensor Factor Analysis for Image Restoration
• Adaptive Learning of Tensor Network Structures
• A Statistically Identifiable Model for Tensor-Valued Gaussian Random Variables
• Early soft and flexible fusion of EEG and fMRI via tensor decompositions
• Computing linear sections of varieties: quantum entanglement, tensor decompositions and beyond
• Robust Factorization and Completion of Streaming Tensor Data via Variational Bayesian Inference
• Generating coupled cluster code for modern distributed memory tensor software
• Comparison of Accuracy and Scalability of Gauss-Newton and Alternating Least Squares for CP Decomposition
• PLANC: Parallel Low Rank Approximation with Non-negativity Constraints
• SNAP: Finding Approximate Second-Order Stationary Solutions Efficiently for Non-convex Linearly Constrained Problems
• Tensor completion using geodesics on Segre manifolds
• Regularized and Smooth Double Core Tensor Factorization for Heterogeneous Data
• Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization
• Sparse MTTKRP Acceleration for Tensor Decomposition on GPU
• A tensor model for the calibration of air-coupled ultrasonic sensor arrays in 3D imaging
• Efficient Constrained Tensor Factorization by Alternating Optimization with Primal-Dual Splitting
• The average condition number of most tensor rank decomposition problems is infinite
• Multi-linear Tensor Autoregressive Models
• Low-Rank Temporal Attention-Augmented Bilinear Network for financial time-series forecasting
• Schrödinger PCA: On the Duality between Principal Component Analysis and Schrödinger Equation
• Tensorized Random Projections
• Several Approximation Algorithms for Sparse Best Rank-1 Approximation to Higher-Order Tensors
• On the largest multilinear singular values of higher-order tensors
• Uncovering migration systems through spatio-temporal tensor co-clustering
• Concurrent Alternating Least Squares for multiple simultaneous Canonical Polyadic Decompositions
• Image Classification by Throwing Quantum Kitchen Sinks at Tensor Networks
• Minimizing low-rank models of high-order tensors: Hardness, span, tight relaxation, and applications
• tHoops: A Multi-Aspect Analytical Framework Spatio-Temporal Basketball Data
• On Recoverability of Randomly Compressed Tensors with Low CP Rank
• SliceNStitch: Continuous CP Decomposition of Sparse Tensor Streams
• LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search
• MTC: Multiresolution Tensor Completion from Partial and Coarse Observations
• Shared Memory Parallelization of MTTKRP for Dense Tensors
• TensorKrowch: Smooth integration of tensor networks in machine learning
• A Normal Form Algorithm for Tensor Rank Decomposition
• The landscape of software for tensor computations
• Multi-Slice Low-Rank Tensor Decomposition Based Multi-Atlas Segmentation: Application to Automatic Pathological Liver CT Segmentation
• Stencil Computations on AMD and Nvidia Graphics Processors: Performance and Tuning Strategies
• Marginalizable Density Models
• KNH: Multi-View Modeling with K-Nearest Hyperplanes Graph for Misinformation Detection
• Spatial-Temporal Tensor Graph Convolutional Network for Traffic Prediction
• Tensor Valued Common and Individual Feature Extraction: Multi-dimensional Perspective
• Message passing-based joint CFO and channel estimation in millimeter wave systems with one-bit ADCs
• On the average condition number of tensor rank decompositions
• TensorMap: Lidar-Based Topological Mapping and Localization via Tensor Decompositions
• A Random Matrix Perspective on Random Tensors
• Robust Low-tubal-rank Tensor Completion based on Tensor Factorization and Maximum Correntopy Criterion
• Nesterov Acceleration of Alternating Least Squares for Canonical Tensor Decomposition: Momentum Step Size Selection and Restart Mechanisms
• Network Anomaly Detection based on Tensor Decomposition
• Hyperspectral Super-resolution: A Coupled Nonnegative Block-term Tensor Decomposition Approach
• Derivation and Analysis of Fast Bilinear Algorithms for Convolution
• Completing a joint PMF from projections: a low-rank coupled tensor factorization approach
• Sample, computation vs storage tradeoffs for classification using tensor subspace models
• Tensor-Based Parameter Estimation of Double Directional Massive MIMO Channel with Dual-Polarized Antennas
• Robust Tensor Principal Component Analysis: Exact Recovery via Deterministic Model
• Nonlinear Algebra and Applications
• Convolutional Neural Networks with Transformed Input based on Robust Tensor Network Decomposition
• The condition number of many tensor decompositions is invariant under Tucker compression
• Bayesian Allocation Model: Inference by Sequential Monte Carlo for Nonnegative Tensor Factorizations and Topic Models using Polya Urns
• Fast Low-Rank Tensor Decomposition by Ridge Leverage Score Sampling
• Towards Time-Aware Context-Aware Deep Trust Prediction in Online Social Networks
• Resonator Networks outperform optimization methods at solving high-dimensional vector factorization
• Long Random Matrices and Tensor Unfolding
• Near Optimal Sketching of Low-Rank Tensor Regression
• On the rank and the approximation of symmetric tensors
• Adaptive Anomaly Detection in Network Flows with Low-Rank Tensor Decompositions and Deep Unrolling
• Low-Rank and Sparse Enhanced Tucker Decomposition for Tensor Completion
• PASTA: A Parallel Sparse Tensor Algorithm Benchmark Suite
• Recovery of Joint Probability Distribution from one-way marginals: Low rank Tensors and Random Projections
• Reinforcement Learning in Factored Action Spaces using Tensor Decompositions
• Large-Scale Spectrum Occupancy Learning via Tensor Decomposition and LSTM Networks
• Novel Algorithms based on Majorization Minimization for Nonnegative Matrix Factorization
• ISLET: Fast and Optimal Low-rank Tensor Regression via Importance Sketching
• Toward a generalization of Kruskal's theorem on tensor decomposition
• On the minimal algebraic complexity of the rank-one approximation problem for general inner products
• Variational Bayesian inference for CP tensor completion with side information
• A quadratically convergent proximal algorithm for nonnegative tensor decomposition
• Tensor Completion via Tensor Networks with a Tucker Wrapper
• Incremental View Maintenance with Triple Lock Factorization Benefits
• The Epsilon-Alternating Least Squares for Orthogonal Low-Rank Tensor Approximation and Its Global Convergence
• Tight Lower Bound on the Tensor Rank based on the Maximally Square Unfolding
• Multilinear SVD for Millimeter Wave Channel Parameter Estimation
• System-Level Performance Modeling of Photonic In-Memory Computing
• MERACLE: Constructive layer-wise conversion of a Tensor Train into a MERA
• Universality of the Wigner-Gurau limit for random tensors
• Correlation-based Initialization Algorithm for Tensor-based HSI Compression Methods
• HOSVD-Based Algorithm for Weighted Tensor Completion
• T-SVD Based Non-convex Tensor Completion and Robust Principal Component Analysis
• Search-free DOA Estimation Method Based on Tensor Decomposition and Polynomial Rooting for Transmit Beamspace MIMO Radar
• GRATE: Granular Recovery of Aggregated Tensor Data by Example
• Analysing Global Fixed Income Markets with Tensors
• Parallel Sparse Tensor Decomposition in Chapel
• Analyzing Political Text at Scale with Online Tensor LDA
• Property Inheritance for Subtensors in Tensor Train Decompositions
• Deepfake Representation with Multilinear Regression
• Multi-Tensor Network Representation for High-Order Tensor Completion
• Latent Functional PARAFAC for modeling multidimensional longitudinal data
• Multipath CNN with alpha matte inference for knee tissue segmentation from MRI
• Generalized Visual Information Analysis via Tensorial Algebra
• On the convergence of Jacobi-type algorithms for Independent Component Analysis
• On characteristic rank for matrix and tensor completion
• Large-Scale Learning with Fourier Features and Tensor Decompositions
• Tensor Methods for Generating Compact Uncertainty Quantification and Deep Learning Models
• Multivariate Convolutional Sparse Coding with Low Rank Tensor
• Landscape analysis of an improved power method for tensor decomposition
• REP: Predicting the Time-Course of Drug Sensitivity
• The decompositions and positive semidefiniteness of fourth-order conjugate partial-symmetric tensors with applications
• Protecting Big Data Privacy Using Randomized Tensor Network Decomposition and Dispersed Tensor Computation
• A study of the Multicriteria decision analysis based on the time-series features and a TOPSIS method proposal for a tensorial approach
• Rademacher Random Projections with Tensor Networks
• CP-MDP: A CANDECOMP-PARAFAC Decomposition Approach to Solve a Markov Decision Process Multidimensional Problem
• DynACPD Embedding Algorithm for Prediction Tasks in Dynamic Networks
• Hanson-Wright Inequality for Random Tensors under Einstein Product
• Deep Latent Factor Model for Collaborative Filtering
• On approximate diagonalization of third order symmetric tensors by orthogonal transformations
• Learning Fair Canonical Polyadical Decompositions using a Kernel Independence Criterion
• Riemannian preconditioned coordinate descent for low multi-linear rank approximation
• A Parallel Sparse Tensor Benchmark Suite on CPUs and GPUs
• Jacobi-type algorithms for homogeneous polynomial optimization on Stiefel manifolds with applications to tensor approximations
• Tucker Tensor Decomposition on FPGA
• Nonnegative Low Rank Tensor Approximation and its Application to Multi-dimensional Images
• New penalized criteria for smooth non-negative tensor factorization with missing entries
• Noisy Tensor Completion for Tensors with a Sparse Canonical Polyadic Factor
• CoCoTen: Detecting Adversarial Inputs to Large Language Models through Latent Space Features of Contextual Co-occurrence Tensors
• Tensor Learning-based Precoder Codebooks for FD-MIMO Systems
• Tensor-based framework for training flexible neural networks
• Efficient Alternating Least Squares Algorithms for Low Multilinear Rank Approximation of Tensors