Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1

Abstract

Machine learning and data mining algorithms are becoming increasingly important in analyzing large volume, multi-relational and multi--modal datasets, which are often conveniently represented as multiway arrays or tensors. It is therefore timely and valuable for the multidisciplinary research community to review tensor decompositions and tensor networks as emerging tools for large-scale data analysis and data mining. We provide the mathematical and graphical representations and interpretation of tensor networks, with the main focus on the Tucker and Tensor Train (TT) decompositions and their extensions or generalizations. Keywords: Tensor networks, Function-related tensors, CP decomposition, Tucker models, tensor train (TT) decompositions, matrix product states (MPS), matrix product operators (MPO), basic tensor operations, multiway component analysis, multilinear blind source separation, tensor completion, linear/multilinear dimensionality reduction, large-scale optimization problems, symmetric eigenvalue decomposition (EVD), PCA/SVD, huge systems of linear equations, pseudo-inverse of very large matrices, Lasso and Canonical Correlation Analysis (CCA) (This is Part 1)

176 pages

References in corpus (8)

• Convex Optimization for Big Data
• Parallel Tensor Compression for Large-Scale Scientific Data
• Tensor Spectral Clustering for Partitioning Higher-order Network Structures
• Quantum MERA Channels
• DFacTo: Distributed Factorization of Tensors
• Stable, Robust and Super Fast Reconstruction of Tensors Using Multi-Way Projections
• A reduced basis approach for calculation of the Bethe-Salpeter excitation energies using low-rank tensor factorizations
• Improving the numerical stability of fast matrix multiplication

Cited by in corpus (56)

• The ITensor Software Library for Tensor Network Calculations
• Unsupervised Generative Modeling Using Matrix Product States
• Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
• Hyper-optimized tensor network contraction
• Lecture Notes of Tensor Network Contractions
• Quantum principal component analysis only achieves an exponential speedup because of its state preparation assumptions
• Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure
• Robust Low-Rank Tensor Ring Completion
• Contracting Arbitrary Tensor Networks: General Approximate Algorithm and Applications in Graphical Models and Quantum Circuit Simulations
• Multidimensional approximation of nonlinear dynamical systems
• Provable Tensor Ring Completion
• The Presence and Absence of Barren Plateaus in Tensor-network Based Machine Learning
• High--Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality
• Generative Tensor Network Classification Model for Supervised Machine Learning
• Fast counting with tensor networks
• Tensor networks for quantum machine learning
• Tensor Train Neighborhood Preserving Embedding
• A tensor network view of multilayer multiconfiguration time-dependent Hartree methods
• Existence of a Spectral Gap in the Affleck-Kennedy-Lieb-Tasaki Model on the Hexagonal Lattice
• Tensor networks for unsupervised machine learning
• Is Deep Learning a Renormalization Group Flow?
• Gauging tensor networks with belief propagation
• Kronecker CP Decomposition with Fast Multiplication for Compressing RNNs
• A literature survey of matrix methods for data science
• Quantum Machine Learning Tensor Network States
• A Dimensionality Reduction Approach for Convolutional Neural Networks
• Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression
• Reduced-Rank Tensor-on-Tensor Regression and Tensor-variate Analysis of Variance
• Tangent-Space Gradient Optimization of Tensor Network for Machine Learning
• Learning with tree tensor networks: complexity estimates and model selection
• Opportunities and challenges of quantum computing for climate modelling
• On the descriptive power of Neural-Networks as constrained Tensor Networks with exponentially large bond dimension
• Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions -- Part I
• Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions -- Part II
• Undecidability in Physics: a Review
• Deep convolutional tensor network
• Avoidance, Adjacency, and Association in Distributed Systems Design
• Prospects of tensor-based numerical modeling of the collective electrostatic potential in many-particle systems
• The Linear Span of Uniform Matrix Product States
• Principal Component Analysis with Tensor Train Subspace
• Sampling-Free Learning of Bayesian Quantized Neural Networks
• Survey on Computational Applications of Tensor Network Simulations
• Fast gradient-free activation maximization for neurons in spiking neural networks
• Lightweight Attribute Localizing Models for Pedestrian Attribute Recognition
• Representation Theorem for Matrix Product States
• Targeted Advertising on Social Networks Using Online Variational Tensor Regression
• On the Optimal Linear Contraction Order of Tree Tensor Networks, and Beyond
• Scalable Community Detection Using Quantum Hamiltonian Descent and QUBO Formulation
• PASTA: A Parallel Sparse Tensor Algorithm Benchmark Suite
• Tensor Network Estimation of Distribution Algorithms
• Tensor-SqRA: Modeling the Transition Rates of Interacting Molecular Systems in terms of Potential Energies
• Nonlinear Tensor Ring Network
• New perspectives on quantum kernels through the lens of entangled tensor kernels
• Multi-Tensor Network Representation for High-Order Tensor Completion
• Patch-based Medical Image Segmentation using Matrix Product State Tensor Networks
• A Parallel Sparse Tensor Benchmark Suite on CPUs and GPUs