Variational Tensor Neural Networks for Deep Learning
arXiv:2211.14657 · doi:10.1038/s41598-024-69366-8
Abstract
Deep neural networks (NNs) encounter scalability limitations when confronted with a vast array of neurons, thereby constraining their achievable network depth. To address this challenge, we propose an integration of tensor networks (TN) into NN frameworks, combined with a variational DMRG-inspired training technique. This in turn, results in a scalable tensor neural network (TNN) architecture capable of efficient training over a large parameter space. Our variational algorithm utilizes a local gradient-descent technique, enabling manual or automatic computation of tensor gradients, facilitating design of hybrid TNN models with combined dense and tensor layers. Our training algorithm further provides insight on the entanglement structure of the tensorized trainable weights and correlation among the model parameters. We validate the accuracy and efficiency of our method by designing TNN models and providing benchmark results for linear and non-linear regressions, data classification and image recognition on MNIST handwritten digits.
21 pages, 19 figures
References in corpus (24)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Efficient classical simulation of slightly entangled quantum computations
- A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States
- Deep Learning for Hyperspectral Image Classification: An Overview
- Matrix Product States, Projected Entangled Pair States, and variational renormalization group methods for quantum spin systems
- Efficient simulation of one-dimensional quantum many-body systems
- Machine learning phases of matter
- A high-bias, low-variance introduction to Machine Learning for physicists
- The iTEBD algorithm beyond unitary evolution
- Machine learning quantum phases of matter beyond the fermion sign problem
- A Comprehensive Analysis of Deep Regression
- A Review of automatic differentiation and its efficient implementation
- Tensor Methods in Computer Vision and Deep Learning
- Detection of phase transition via convolutional neural network
- Learning Relevant Features of Data with Multi-scale Tensor Networks
- Neural tensor contractions and the expressive power of deep neural quantum states
- Machine Learning for Condensed Matter Physics
- An end-to-end trainable hybrid classical-quantum classifier
- Unsupervised identification of topological order using predictive models
- A Model Compression Method with Matrix Product Operators for Speech Enhancement
- Randomness-enhanced expressivity of quantum neural networks
- CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks
- Tensor network compressibility of convolutional models
- Boosting Defect Detection in Manufacturing using Tensor Convolutional Neural Networks