Residual Matrix Product State for Machine Learning

Abstract

Tensor network, which originates from quantum physics, is emerging as an efficient tool for classical and quantum machine learning. Nevertheless, there still exists a considerable accuracy gap between tensor network and the sophisticated neural network models for classical machine learning. In this work, we combine the ideas of matrix product state (MPS), the simplest tensor network structure, and residual neural network and propose the residual matrix product state (ResMPS). The ResMPS can be treated as a network where its layers map the "hidden" features to the outputs (e.g., classifications), and the variational parameters of the layers are the functions of the features of the samples (e.g., pixels of images). This is different from neural network, where the layers map feed-forwardly the features to the output. The ResMPS can equip with the non-linear activations and dropout layers, and outperforms the state-of-the-art tensor network models in terms of efficiency, stability, and expression power. Besides, ResMPS is interpretable from the perspective of polynomial expansion, where the factorization and exponential machines naturally emerge. Our work contributes to connecting and hybridizing neural and tensor networks, which is crucial to further enhance our understand of the working mechanisms and improve the performance of both models.

References in corpus (23)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• The density-matrix renormalization group in the age of matrix product states
• Deep Learning using Rectified Linear Units (ReLU)
• Recurrent Neural Network Regularization
• Matrix Product States, Projected Entangled Pair States, and variational renormalization group methods for quantum spin systems
• Tensor network states and geometry
• Unsupervised Generative Modeling Using Matrix Product States
• Equivalence of restricted Boltzmann machines and tensor network states
• Quantum Entanglement in Deep Learning Architectures
• Lecture Notes of Tensor Network Contractions
• Experimental Machine Learning of Quantum States
• Tree Tensor Networks for Generative Modeling
• Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure
• Learning Relevant Features of Data with Multi-scale Tensor Networks
• TensorNetwork for Machine Learning
• Supervised Learning with Projected Entangled Pair States
• Entanglement and tensor network states
• A Quantum Many-body Wave Function Inspired Language Modeling Approach
• Compressing deep neural networks by matrix product operators
• Tangent-Space Gradient Optimization of Tensor Network for Machine Learning
• Deep convolutional tensor network
• Quantum-Classical Machine learning by Hybrid Tensor Networks
• Bayesian Tensor Network with Polynomial Complexity for Probabilistic Machine Learning

Cited by in corpus (2)

• Tensor networks for interpretable and efficient quantum-inspired machine learning
• Universal scaling laws in quantum-probabilistic machine learning by tensor network towards interpreting representation and generalization powers