Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives

Abstract

Part 2 of this monograph builds on the introduction to tensor networks and their operations presented in Part 1. It focuses on tensor network models for super-compressed higher-order representation of data/parameters and related cost functions, while providing an outline of their applications in machine learning and data analytics. A particular emphasis is on the tensor train (TT) and Hierarchical Tucker (HT) decompositions, and their physically meaningful interpretations which reflect the scalability of the tensor network approach. Through a graphical approach, we also elucidate how, by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volumes of data/parameters, thereby alleviating or even eliminating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification (support tensor machines, canonical correlation analysis, higher order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and in the optimization of deep neural networks. Part 1 and Part 2 of this work can be used either as stand-alone separate texts, or indeed as a conjoint comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions.

232 pages

References in corpus (13)

• Deep Learning in Neural Networks: An Overview
• A class of quantum many-body states that can be efficiently simulated
• The iTEBD algorithm beyond unitary evolution
• Experimental Realization of Quantum Artificial Intelligence
• A Strictly Single-Site DMRG Algorithm with Subspace Expansion
• Generic Construction of Efficient Matrix Product Operators
• Ultimate tensorization: compressing convolutional and FC layers alike
• DFacTo: Distributed Factorization of Tensors
• Stable, Robust and Super Fast Reconstruction of Tensors Using Multi-Way Projections
• Learning from Multiway Data: Simple and Efficient Tensor Regression
• Tucker Tensor Regression and Neuroimaging Analysis
• Nested Expectation Propagation for Gaussian Process Classification with a Multinomial Probit Likelihood
• Analytic Optimization of a MERA network and its Relevance to Quantum Integrability and Wavelet

Cited by in corpus (47)

• Hyper-optimized tensor network contraction
• Tensor Methods in Computer Vision and Deep Learning
• Machine Learning for Quantum Matter
• Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure
• Robust Low-Rank Tensor Ring Completion
• The Presence and Absence of Barren Plateaus in Tensor-network Based Machine Learning
• Generative Tensor Network Classification Model for Supervised Machine Learning
• Fast counting with tensor networks
• Existence of a Spectral Gap in the Affleck-Kennedy-Lieb-Tasaki Model on the Hexagonal Lattice
• Is Deep Learning a Renormalization Group Flow?
• Gauging tensor networks with belief propagation
• Tree tensor network classifiers for machine learning: from quantum-inspired to quantum-assisted
• Generative machine learning with tensor networks: benchmarks on near-term quantum computers
• An adaptive algorithm for quantum circuit simulation
• Wide Compression: Tensor Ring Nets
• Correlation length in random MPS and PEPS
• A literature survey of matrix methods for data science
• Quantum Machine Learning Tensor Network States
• A Dimensionality Reduction Approach for Convolutional Neural Networks
• Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression
• Tangent-Space Gradient Optimization of Tensor Network for Machine Learning
• Learning with tree tensor networks: complexity estimates and model selection
• Higher-dimension Tensor Completion via Low-rank Tensor Ring Decomposition
• Low-rank nonnegative tensor approximation via alternating projections and sketching
• The minimal canonical form of a tensor network
• Learning Effective Spin Hamiltonian of Quantum Magnet
• SGD_Tucker: A Novel Stochastic Optimization Strategy for Parallel Sparse Tucker Decomposition
• On the descriptive power of Neural-Networks as constrained Tensor Networks with exponentially large bond dimension
• Brain correlates of task-load and dementia elucidation with tensor machine learning using oddball BCI paradigm
• Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions -- Part II
• The trouble with tensor ring decompositions
• Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions -- Part I
• Deep convolutional tensor network
• Avoidance, Adjacency, and Association in Distributed Systems Design
• Equivariant Tensor Network Potentials
• The resource theory of tensor networks
• Survey on Computational Applications of Tensor Network Simulations
• Marginalizable Density Models
• Sampling-Free Learning of Bayesian Quantized Neural Networks
• Computing low-rank approximations of large-scale matrices with the Tensor Network randomized SVD
• Fast gradient-free activation maximization for neurons in spiking neural networks
• Exact average many-body interatomic interaction model for random alloys
• On the Optimal Linear Contraction Order of Tree Tensor Networks, and Beyond
• Tensor Network Estimation of Distribution Algorithms
• Representation Theorem for Matrix Product States
• MERACLE: Constructive layer-wise conversion of a Tensor Train into a MERA
• A Sampling-Based Method for Tensor Ring Decomposition