Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis

Abstract

The widespread use of multi-sensor technology and the emergence of big datasets has highlighted the limitations of standard flat-view matrix models and the necessity to move towards more versatile data analysis tools. We show that higher-order tensors (i.e., multiway arrays) enable such a fundamental paradigm shift towards models that are essentially polynomial and whose uniqueness, unlike the matrix methods, is guaranteed under verymild and natural conditions. Benefiting fromthe power ofmultilinear algebra as theirmathematical backbone, data analysis techniques using tensor decompositions are shown to have great flexibility in the choice of constraints that match data properties, and to find more general latent components in the data than matrix-based methods. A comprehensive introduction to tensor decompositions is provided from a signal processing perspective, starting from the algebraic foundations, via basic Canonical Polyadic and Tucker models, through to advanced cause-effect and multi-view data analysis schemes. We show that tensor decompositions enable natural generalizations of some commonly used signal processing paradigms, such as canonical correlation and subspace techniques, signal separation, linear regression, feature extraction and classification. We also cover computational aspects, and point out how ideas from compressed sensing and scientific computing may be used for addressing the otherwise unmanageable storage and manipulation problems associated with big datasets. The concepts are supported by illustrative real world case studies illuminating the benefits of the tensor framework, as efficient and promising tools for modern signal processing, data analysis and machine learning applications; these benefits also extend to vector/matrix data through tensorization. Keywords: ICA, NMF, CPD, Tucker decomposition, HOSVD, tensor networks, Tensor Train.

References in corpus (1)

• Era of Big Data Processing: A New Approach via Tensor Networks and Tensor Decompositions

Cited by in corpus (159)

• Tensor Decomposition for Signal Processing and Machine Learning
• Tensor Ring Decomposition
• Channel Estimation for Intelligent Reflecting Surface Assisted MIMO Systems: A Tensor Modeling Approach
• Era of Big Data Processing: A New Approach via Tensor Networks and Tensor Decompositions
• TensorLy: Tensor Learning in Python
• Linked Component Analysis from Matrices to High Order Tensors: Applications to Biomedical Data
• Tensor Methods in Computer Vision and Deep Learning
• High-order tensor flow processing using integrated photonic circuits
• Signal Processing on Higher-Order Networks: Livin' on the Edge ... and Beyond
• Guaranteed Tensor Recovery Fused Low-rankness and Smoothness
• Parallel Algorithms for Constrained Tensor Factorization via the Alternating Direction Method of Multipliers
• Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights
• Bayesian Low Rank Tensor Ring Model for Image Completion
• Generic uniqueness conditions for the canonical polyadic decomposition and INDSCAL
• Tensor Regression Networks
• Introducing Hypergraph Signal Processing: Theoretical Foundation and Practical Applications
• Robust Low-Rank Tensor Ring Completion
• DOA Estimation for Transmit Beamspace MIMO Radar via Tensor Decomposition with Vandermonde Factor Matrix
• Coupled Tensor Decomposition for Hyperspectral and Multispectral Image Fusion with Inter-Image Variability
• Multidimensional approximation of nonlinear dynamical systems
• Provable Tensor Ring Completion
• Decoupling Multivariate Polynomials Using First-Order Information
• Double Coupled Canonical Polyadic Decomposition for Joint Blind Source Separation
• Statistical and computational phase transitions in spiked tensor estimation
• Low-tubal-rank Tensor Completion using Alternating Minimization
• Tensor Computation: A New Framework for High-Dimensional Problems in EDA
• Tensor Networks for Big Data Analytics and Large-Scale Optimization Problems
• Brain-Computer Interface with Corrupted EEG Data: A Tensor Completion Approach
• Sparse Sampling for Inverse Problems with Tensors
• Efficient Tensor Robust PCA under Hybrid Model of Tucker and Tensor Train
• Object-based Multipass InSAR via Robust Low Rank Tensor Decomposition
• Multilinear Compressive Learning
• Very Large-Scale Singular Value Decomposition Using Tensor Train Networks
• Computing Large-Scale Matrix and Tensor Decomposition with Structured Factors: A Unified Nonconvex Optimization Perspective
• An Iterative Reweighted Method for Tucker Decomposition of Incomplete Multiway Tensors
• Blind Direction-of-Arrival Estimation in Acoustic Vector-Sensor Arrays via Tensor Decomposition and Kullback-Leibler Divergence Covariance Fitting
• Matrix Product State for Higher-Order Tensor Compression and Classification
• Compressing 3DCNNs Based on Tensor Train Decomposition
• Muscle Activity Analysis using Higher-Order Tensor Models: Application to Muscle Synergy Identification
• Generic uniqueness of a structured matrix factorization and applications in blind source separation
• Near-Field Localization and Sensing with Large-Aperture Arrays: From Signal Modeling to Processing
• A Low-rank Tensor Regularization Strategy for Hyperspectral Unmixing
• Multi-way Graph Signal Processing on Tensors: Integrative analysis of irregular geometries
• Projection-based QLP Algorithm for Efficiently Computing Low-Rank Approximation of Matrices
• Ergodic Exploration using Tensor Train: Applications in Insertion Tasks
• Dictionary-based Tensor Canonical Polyadic Decomposition
• Reconstruction by Calibration over Tensors for Multi-Coil Multi-Acquisition Balanced SSFP Imaging
• Application of Compressive Sensing Techniques in Distributed Sensor Networks: A Survey
• Reduced-Rank Tensor-on-Tensor Regression and Tensor-variate Analysis of Variance
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• High-dimensional rank-one nonsymmetric matrix decomposition: the spherical case
• Stochastic Mirror Descent for Low-Rank Tensor Decomposition Under Non-Euclidean Losses
• Equivariant minimax dominators of the MLE in the array normal model
• Probabilistic low-rank factorization accelerates tensor network simulations of critical quantum many-body ground states
• Legendre Decomposition for Tensors
• Accelerating Block Coordinate Descent for Nonnegative Tensor Factorization
• New Riemannian preconditioned algorithms for tensor completion via polyadic decomposition
• A Sharp Blockwise Tensor Perturbation Bound for Orthogonal Iteration
• A Comprehensive Comparison of Multi-Dimensional Image Denoising Methods
• Robust Tensor Completion Using Transformed Tensor SVD
• Jacobi-type algorithm for low rank orthogonal approximation of symmetric tensors and its convergence analysis
• A generalization of Kruskal's theorem on tensor decomposition
• Low-rank Tensor Grid for Image Completion
• Consistency of Muscle Synergies Extracted via Higher-Order Tensor Decomposition Towards Myoelectric Control
• Power Iteration for Tensor PCA
• On the use of higher-order tensors to model muscle synergies
• A higher-order LQ decomposition for separable covariance models
• Shortcut Matrix Product States and its applications
• Mutual information for low-rank even-order symmetric tensor estimation
• Computing linear sections of varieties: quantum entanglement, tensor decompositions and beyond
• Higher order tensor decomposition for proportional myoelectric control based on muscle synergies
• Regularized and Smooth Double Core Tensor Factorization for Heterogeneous Data
• Tensor Decompositions in Deep Learning
• Multiresolution Tensor Decomposition for Multiple Spatial Passing Networks
• A tensor model for the calibration of air-coupled ultrasonic sensor arrays in 3D imaging
• CNN Acceleration by Low-rank Approximation with Quantized Factors
• An Efficient and Flexible Spike Train Model via Empirical Bayes
• Tensor Low Rank Modeling and Its Applications in Signal Processing
• Efficient Constrained Tensor Factorization by Alternating Optimization with Primal-Dual Splitting
• Superfast CUR Matrix Algorithms, Their Pre-Processing and Extensions
• On the largest multilinear singular values of higher-order tensors
• Several Approximation Algorithms for Sparse Best Rank-1 Approximation to Higher-Order Tensors
• Efficient Low Rank Tensor Ring Completion
• Speech Artifact Removal from EEG Recordings of Spoken Word Production with Tensor Decomposition
• Image Classification by Throwing Quantum Kitchen Sinks at Tensor Networks
• Matrix Product State for Feature Extraction of Higher-Order Tensors
• Adaptive Higher-order Spectral Estimators
• Learning Good State and Action Representations via Tensor Decomposition
• Robust Multi-dimensional Model Order Estimation Using LineAr Regression of Global Eigenvalues (LaRGE)
• The landscape of software for tensor computations
• Third-Order Statistics Reconstruction from Compressive Measurements
• Randomized algorithms for the low multilinear rank approximations of tensors
• Multi-Graph Tensor Networks
• Tensor Valued Common and Individual Feature Extraction: Multi-dimensional Perspective
• Multi-Slice Low-Rank Tensor Decomposition Based Multi-Atlas Segmentation: Application to Automatic Pathological Liver CT Segmentation
• A Unified Framework for Coupled Tensor Completion
• A review of heterogeneous data mining for brain disorders
• Robust Low-tubal-rank Tensor Completion based on Tensor Factorization and Maximum Correntopy Criterion
• Tensor Networks for Multi-Modal Non-Euclidean Data
• FasTer: Fast Tensor Completion with Nonconvex Regularization
• Nonlinear Algebra and Applications
• Long Random Matrices and Tensor Unfolding
• Frequency-Weighted Robust Tensor Principal Component Analysis
• Implicit Regularization and Entrywise Convergence of Riemannian Optimization for Low Tucker-Rank Tensor Completion
• Convolutional Neural Networks with Transformed Input based on Robust Tensor Network Decomposition
• Channel Estimation for Millimeter Wave Multiuser MIMO Systems via PARAFAC Decomposition
• On the rank and the approximation of symmetric tensors
• Finding a low-rank basis in a matrix subspace
• Online Multilinear Dictionary Learning
• Multi-mode Core Tensor Factorization based Low-Rankness and Its Applications to Tensor Completion
• Representation Theorem for Matrix Product States
• Quintic-scaling rank-reduced coupled cluster theory with single and double excitations
• Two-Dimensional DOA Estimation for L-shaped Nested Array via Tensor Modeling
• Online adaptive group-wise sparse NPLS for ECoG neural signal decoding
• Variational Bayesian inference for CP tensor completion with side information
• A quadratically convergent proximal algorithm for nonnegative tensor decomposition
• ISLET: Fast and Optimal Low-rank Tensor Regression via Importance Sketching
• T-SVD Based Non-convex Tensor Completion and Robust Principal Component Analysis
• Reducing Computational Complexity of Tensor Contractions via Tensor-Train Networks
• Toward a generalization of Kruskal's theorem on tensor decomposition
• Deep Compression of Sum-Product Networks on Tensor Networks
• Fast tensorial JADE
• Tensor Network Kalman Filtering for Large-Scale LS-SVMs
• Tensor Completion via Tensor Networks with a Tucker Wrapper
• The Epsilon-Alternating Least Squares for Orthogonal Low-Rank Tensor Approximation and Its Global Convergence
• Sparse and redundant signal representations for x-ray computed tomography
• MERACLE: Constructive layer-wise conversion of a Tensor Train into a MERA
• PASTA: A Parallel Sparse Tensor Algorithm Benchmark Suite
• Low-Rank Tensor Decomposition-Aided Channel Estimation for Millimeter Wave MIMO-OFDM Systems
• On approximate diagonalization of third order symmetric tensors by orthogonal transformations
• Sparse Representation of 3D Images for Piecewise Dimensionality Reduction with High Quality Reconstruction
• Bond type restricted radial distribution functions for accurate machine learning prediction of atomization energies
• Globally convergent Jacobi-type algorithms for simultaneous orthogonal symmetric tensor diagonalization
• Low Tensor Train- and Low Multilinear Rank Approximations for De-speckling and Compression of 3D Optical Coherence Tomography Images
• Tensor-based Multi-dimensional Wideband Channel Estimation for mmWave Hybrid Cylindrical Arrays
• Modelling hidden structure of signals in group data analysis with modified (Lr, 1) and block-term decompositions
• Attn-HybridNet: Improving Discriminability of Hybrid Features with Attention Fusion
• Tensor p-shrinkage nuclear norm for low-rank tensor completion
• QRP Variation of Cross--Approximation Iterations for Low Rank Approximation
• Higher-Order Block Term Decomposition for Spatially Folded fMRI Data
• A study of the Multicriteria decision analysis based on the time-series features and a TOPSIS method proposal for a tensorial approach
• Multi-Tensor Network Representation for High-Order Tensor Completion
• Fast Hypergraph Regularized Nonnegative Tensor Ring Factorization Based on Low-Rank Approximation
• Multivariate Convolutional Sparse Coding with Low Rank Tensor
• HOTCAKE: Higher Order Tucker Articulated Kernels for Deeper CNN Compression
• Tensor-Train Parameterization for Ultra Dimensionality Reduction
• Tensor Methods for Generating Compact Uncertainty Quantification and Deep Learning Models
• On the convergence of Jacobi-type algorithms for Independent Component Analysis
• Tensor Learning-based Precoder Codebooks for FD-MIMO Systems
• Property Inheritance for Subtensors in Tensor Train Decompositions
• HOTTBOX: Higher Order Tensor ToolBOX
• Jacobi-type algorithms for homogeneous polynomial optimization on Stiefel manifolds with applications to tensor approximations
• Trace maximization algorithm for the approximate tensor diagonalization
• MiSC: Mixed Strategies Crowdsourcing
• Tensor Matched Subspace Detection
• Finite Differences in Forward and Inverse Imaging Problems--MaxPol Design
• Protecting Big Data Privacy Using Randomized Tensor Network Decomposition and Dispersed Tensor Computation
• Iterative Block Tensor Singular Value Thresholding for Extraction of Low Rank Component of Image Data
• Fast Randomized Algorithms for t-Product Based Tensor Operations and Decompositions with Applications to Imaging Data