Era of Big Data Processing: A New Approach via Tensor Networks and Tensor Decompositions

Abstract

Many problems in computational neuroscience, neuroinformatics, pattern/image recognition, signal processing and machine learning generate massive amounts of multidimensional data with multiple aspects and high dimensionality. Tensors (i.e., multi-way arrays) provide often a natural and compact representation for such massive multidimensional data via suitable low-rank approximations. Big data analytics require novel technologies to efficiently process huge datasets within tolerable elapsed times. Such a new emerging technology for multidimensional big data is a multiway analysis via tensor networks (TNs) and tensor decompositions (TDs) which represent tensors by sets of factor (component) matrices and lower-order (core) tensors. Dynamic tensor analysis allows us to discover meaningful hidden structures of complex data and to perform generalizations by capturing multi-linear and multi-aspect relationships. We will discuss some fundamental TN models, their mathematical and graphical descriptions and associated learning algorithms for large-scale TDs and TNs, with many potential applications including: Anomaly detection, feature extraction, classification, cluster analysis, data fusion and integration, pattern recognition, predictive modeling, regression, time series analysis and multiway component analysis. Keywords: Large-scale HOSVD, Tensor decompositions, CPD, Tucker models, Hierarchical Tucker (HT) decomposition, low-rank tensor approximations (LRA), Tensorization/Quantization, tensor train (TT/QTT) - Matrix Product States (MPS), Matrix Product Operator (MPO), DMRG, Strong Kronecker Product (SKP).

Part of this work was presented on the International Workshop on Smart Info-Media Systems in Asia,(invited talk - SISA-2013) Sept.30--Oct.2, 2013, Nagoya, JAPAN

References in corpus (4)

• Matrix Product States, Projected Entangled Pair States, and variational renormalization group methods for quantum spin systems
• Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis
• Computations in Quantum Tensor Networks
• Fundamental Tensor Operations for Large-Scale Data Analysis in Tensor Train Formats

Cited by in corpus (41)

• Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis
• Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1
• Tensor Ring Decomposition
• Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
• Linked Component Analysis from Matrices to High Order Tensors: Applications to Biomedical Data
• Lecture Notes of Tensor Network Contractions
• Enabling High-Dimensional Hierarchical Uncertainty Quantification by ANOVA and Tensor-Train Decomposition
• Tensor Analysis and Fusion of Multimodal Brain Images
• Reduced-order modeling with artificial neurons for gravitational-wave inference
• Riemannian optimization of isometric tensor networks
• The tensor t-function: a definition for functions of third-order tensors
• Tensor Networks for Big Data Analytics and Large-Scale Optimization Problems
• Fourth-order Tensors with Multidimensional Discrete Transforms
• Randomized Tensor Ring Decomposition and Its Application to Large-scale Data Reconstruction
• Fundamental Tensor Operations for Large-Scale Data Analysis in Tensor Train Formats
• Learning Compact Recurrent Neural Networks with Block-Term Tensor Decomposition
• BT-Nets: Simplifying Deep Neural Networks via Block Term Decomposition
• Optimal Low-Rank Tensor Recovery from Separable Measurements: Four Contractions Suffice
• Discovering Hidden Structure in High Dimensional Human Behavioral Data via Tensor Factorization
• Efficient Contraction of Large Tensor Networks for Weighted Model Counting through Graph Decompositions
• Low-rank Tensor Grid for Image Completion
• Tensor network method for reversible classical computation
• TNSPackage: A Fortran2003 library designed for tensor network state methods
• Tensor Train Discrete Grid-Based Filters: Breaking the Curse of Dimensionality
• Principal Component Analysis with Tensor Train Subspace
• Dynamic Spatiotemporal Graph Neural Network with Tensor Network
• Several Approximation Algorithms for Sparse Best Rank-1 Approximation to Higher-Order Tensors
• Infinitely Wide Tensor Networks as Gaussian Process
• Sparse Tensor Algebra Optimizations with Workspaces
• Tensor Valued Common and Individual Feature Extraction: Multi-dimensional Perspective
• Multi-Graph Tensor Networks
• Tensor-Train Networks for Learning Predictive Modeling of Multidimensional Data
• Tensor Networks for Multi-Modal Non-Euclidean Data
• Tensor decompositions and algorithms, with applications to tensor learning
• A Tensor Network Kalman filter with an application in recursive MIMO Volterra system identification
• A Fast Implementation for the Canonical Polyadic Decomposition
• A Parallel Sparse Tensor Benchmark Suite on CPUs and GPUs
• Sparse Hierarchical Tucker Factorization and its Application to Healthcare
• Recurrent Graph Tensor Networks: A Low-Complexity Framework for Modelling High-Dimensional Multi-Way Sequence
• Nonlinear system identification with regularized Tensor Network B-splines
• Block-term Tensor Neural Networks