Reduced-Rank Tensor-on-Tensor Regression and Tensor-variate Analysis of Variance

Abstract

Fitting regression models with many multivariate responses and covariates can be challenging, but such responses and covariates sometimes have tensor-variate structure. We extend the classical multivariate regression model to exploit such structure in two ways: first, we impose four types of low-rank tensor formats on the regression coefficients. Second, we model the errors using the tensor-variate normal distribution that imposes a Kronecker separable format on the covariance matrix. We obtain maximum likelihood estimators via block-relaxation algorithms and derive their computational complexity and asymptotic distributions. Our regression framework enables us to formulate tensor-variate analysis of variance (TANOVA) methodology. This methodology, when applied in a one-way TANOVA layout, enables us to identify cerebral regions significantly associated with the interaction of suicide attempters or non-attemptor ideators and positive-, negative- or death-connoting words in a functional Magnetic Resonance Imaging study. Another application uses three-way TANOVA on the Labeled Faces in the Wild image dataset to distinguish facial characteristics related to ethnic origin, age group and gender.

arXiv comment35 pages, 12 figures, 2 tables, 2 algorithms.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022; in press

References in corpus (15)

• A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States
• Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis
• Area laws in quantum systems: mutual information and correlations
• Tensor Regression with Applications in Neuroimaging Data Analysis
• Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1
• Tensor Ring Decomposition
• Tensor Methods in Computer Vision and Deep Learning
• Multilinear tensor regression for longitudinal relational data
• Tensor-on-tensor regression
• STORE: Sparse Tensor Response Regression and Neuroimaging Analysis
• Classification with the matrix-variate-$t$ distribution
• Tensor Regression Using Low-rank and Sparse Tucker Decompositions
• CP Degeneracy in Tensor Regression
• A higher-order LQ decomposition for separable covariance models
• FAST Adaptive Smoothing and Thresholding for Improved Activation Detection in Low-Signal fMRI