Learning from Multiway Data: Simple and Efficient Tensor Regression

Abstract

Tensor regression has shown to be advantageous in learning tasks with multi-directional relatedness. Given massive multiway data, traditional methods are often too slow to operate on or suffer from memory bottleneck. In this paper, we introduce subsampled tensor projected gradient to solve the problem. Our algorithm is impressively simple and efficient. It is built upon projected gradient method with fast tensor power iterations, leveraging randomized sketching for further acceleration. Theoretical analysis shows that our algorithm converges to the correct solution in fixed number of iterations. The memory requirement grows linearly with the size of the problem. We demonstrate superior empirical performance on both multi-linear multi-task learning and spatio-temporal applications.

10 pages, Proceedings of the 33rd International Conference on Machine Learning (ICML-16), 2016

References in corpus (1)

• Tensor Regression with Applications in Neuroimaging Data Analysis

Cited by in corpus (14)

• Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
• Tensor Methods in Computer Vision and Deep Learning
• Tensor Regression Networks
• Long-term Forecasting using Higher Order Tensor RNNs
• Tensor Regression Using Low-rank and Sparse Tucker Decompositions
• CP Degeneracy in Tensor Regression
• Low-rank Tensor Estimation via Riemannian Gauss-Newton: Statistical Optimality and Second-Order Convergence
• Scalable Spatiotemporally Varying Coefficient Modelling with Bayesian Kernelized Tensor Regression
• Sparse Tensor Additive Regression
• Fast Low-Rank Tensor Decomposition by Ridge Leverage Score Sampling
• Linear Tensor Projection Revealing Nonlinearity
• Multilinear Dirichlet Processes
• Sparse and Low-Rank High-Order Tensor Regression via Parallel Proximal Method
• Fast and Scalable Estimator for Sparse and Unit-Rank Higher-Order Regression Models