Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
arXiv:1708.09165 · doi:10.1561/2200000067
Abstract
Part 2 of this monograph builds on the introduction to tensor networks and their operations presented in Part 1. It focuses on tensor network models for super-compressed higher-order representation of data/parameters and related cost functions, while providing an outline of their applications in machine learning and data analytics. A particular emphasis is on the tensor train (TT) and Hierarchical Tucker (HT) decompositions, and their physically meaningful interpretations which reflect the scalability of the tensor network approach. Through a graphical approach, we also elucidate how, by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volumes of data/parameters, thereby alleviating or even eliminating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification (support tensor machines, canonical correlation analysis, higher order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and in the optimization of deep neural networks. Part 1 and Part 2 of this work can be used either as stand-alone separate texts, or indeed as a conjoint comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions.
arXiv comment232 pages
References in corpus (68)
- Deep Learning in Neural Networks: An Overview
- Quantum Machine Learning
- Entanglement Entropy and Quantum Field Theory
- Area laws for the entanglement entropy - a review
- Efficient classical simulation of slightly entangled quantum computations
- Quantum support vector machine for big data classification
- Quantum state tomography via compressed sensing
- Thermodynamic limit of the density matrix renormalization for the spin-1 Heisenberg chain
- A class of quantum many-body states that can be efficiently simulated
- Manopt, a Matlab toolbox for optimization on manifolds
- Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination
- Tensorizing Neural Networks
- Tensor Regression with Applications in Neuroimaging Data Analysis
- The iTEBD algorithm beyond unitary evolution
- Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1
- Linear Dimensionality Reduction: Survey, Insights, and Generalizations
- Algorithms for entanglement renormalization
- Smooth PARAFAC Decomposition for Tensor Completion
- Experimental Realization of Quantum Artificial Intelligence
- Alternating minimal energy methods for linear systems in higher dimensions. Part I: SPD systems
- Alternating minimal energy methods for linear systems in higher dimensions. Part II: Faster algorithm and application to nonsymmetric systems
- Bayesian Robust Tensor Factorization for Incomplete Multiway Data
- Equivalence of restricted Boltzmann machines and tensor network states
- Density matrix renormalization group algorithms with a single center site
- Era of Big Data Processing: A New Approach via Tensor Networks and Tensor Decompositions
- A Strictly Single-Site DMRG Algorithm with Subspace Expansion
- Density matrix renormalisation group for a quantum spin chain at non-zero temperature
- Generic Construction of Efficient Matrix Product Operators
- On the Expressive Power of Deep Learning: A Tensor Analysis
- Tensor network renormalization yields the multi-scale entanglement renormalization ansatz
- Enabling High-Dimensional Hierarchical Uncertainty Quantification by ANOVA and Tensor-Train Decomposition
- Convex Tensor Decomposition via Structured Schatten Norm Regularization
- Deep Multi-task Representation Learning: A Tensor Factorisation Approach
- Tensor Spectral Clustering for Partitioning Higher-order Network Structures
- Ultimate tensorization: compressing convolutional and FC layers alike
- Computation of extreme eigenvalues in higher dimensions using block tensor train format
- Supervised Learning with Quantum-Inspired Tensor Networks
- DFacTo: Distributed Factorization of Tensors
- Entanglement renormalization and wavelets
- Linear Support Tensor Machine: Pedestrian Detection in Thermal Infrared Images
- Log-Determinant Divergences Revisited: Alpha--Beta and Gamma Log-Det Divergences
- Fast and Guaranteed Tensor Decomposition via Sketching
- A literature survey of low-rank tensor approximation techniques
- Stable, Robust and Super Fast Reconstruction of Tensors Using Multi-Way Projections
- Tensor Networks for Big Data Analytics and Large-Scale Optimization Problems
- A tensor approximation method based on ideal minimal residual formulations for the solution of high-dimensional problems
- Asymptotic behavior of -based Laplacian regularization in semi-supervised learning
- Very Large-Scale Singular Value Decomposition Using Tensor Train Networks
- Exact NMR simulation of protein-size spin systems using tensor train formalism
- Representation and design of wavelets using unitary circuits
- Learning from Multiway Data: Simple and Efficient Tensor Regression
- Efficient tensor completion: Low-rank tensor train
- Deep Convolutional Networks are Hierarchical Kernel Machines
- A reduced basis approach for calculation of the Bethe-Salpeter excitation energies using low-rank tensor factorizations
- Variational Numerical Renormalization Group: Bridging the gap between NRG and Density Matrix Renormalization Group
- Tucker Tensor Regression and Neuroimaging Analysis
- Doubly Decomposing Nonparametric Tensor Regression
- Tensorial Mixture Models
- A generalized Lanczos method for systematic optimization of tensor network states
- Nested Expectation Propagation for Gaussian Process Classification with a Multinomial Probit Likelihood
- Tensor Networks for Latent Variable Analysis. Part I: Algorithms for Tensor Train Decomposition
- Riemannian preconditioning for tensor completion
- A low-rank approach to the computation of path integrals
- Tensor Network alternating linear scheme for MIMO Volterra system identification
- Analytic Optimization of a MERA network and its Relevance to Quantum Integrability and Wavelet
- Matrix Variate RBM and Its Applications
- Riemannian Tensor Completion with Side Information
- Higher-Order Low-Rank Regression
Cited by in corpus (48)
- Hyper-optimized tensor network contraction
- Tensor Methods in Computer Vision and Deep Learning
- Lecture Notes of Tensor Network Contractions
- Machine Learning for Quantum Matter
- Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure
- Robust Low-Rank Tensor Ring Completion
- The Presence and Absence of Barren Plateaus in Tensor-network Based Machine Learning
- Generative Tensor Network Classification Model for Supervised Machine Learning
- Fast counting with tensor networks
- Existence of a Spectral Gap in the Affleck-Kennedy-Lieb-Tasaki Model on the Hexagonal Lattice
- Tree tensor network classifiers for machine learning: from quantum-inspired to quantum-assisted
- Is Deep Learning a Renormalization Group Flow?
- Gauging tensor networks with belief propagation
- Generative machine learning with tensor networks: benchmarks on near-term quantum computers
- An adaptive algorithm for quantum circuit simulation
- Wide Compression: Tensor Ring Nets
- Correlation length in random MPS and PEPS
- A literature survey of matrix methods for data science
- Quantum Machine Learning Tensor Network States
- A Dimensionality Reduction Approach for Convolutional Neural Networks
- Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression
- Learning with tree tensor networks: complexity estimates and model selection
- Tangent-Space Gradient Optimization of Tensor Network for Machine Learning
- Higher-dimension Tensor Completion via Low-rank Tensor Ring Decomposition
- Low-rank nonnegative tensor approximation via alternating projections and sketching
- The minimal canonical form of a tensor network
- Learning Effective Spin Hamiltonian of Quantum Magnet
- SGD_Tucker: A Novel Stochastic Optimization Strategy for Parallel Sparse Tucker Decomposition
- On the descriptive power of Neural-Networks as constrained Tensor Networks with exponentially large bond dimension
- Brain correlates of task-load and dementia elucidation with tensor machine learning using oddball BCI paradigm
- Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions -- Part II
- The trouble with tensor ring decompositions
- Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions -- Part I
- Deep convolutional tensor network
- The resource theory of tensor networks
- Equivariant Tensor Network Potentials
- Avoidance, Adjacency, and Association in Distributed Systems Design
- Marginalizable Density Models
- Computing low-rank approximations of large-scale matrices with the Tensor Network randomized SVD
- Survey on Computational Applications of Tensor Network Simulations
- Sampling-Free Learning of Bayesian Quantized Neural Networks
- Fast gradient-free activation maximization for neurons in spiking neural networks
- Representation Theorem for Matrix Product States
- Tensor Network Estimation of Distribution Algorithms
- Exact average many-body interatomic interaction model for random alloys
- On the Optimal Linear Contraction Order of Tree Tensor Networks, and Beyond
- MERACLE: Constructive layer-wise conversion of a Tensor Train into a MERA
- A Sampling-Based Method for Tensor Ring Decomposition