Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design

Abstract

Deep convolutional networks have witnessed unprecedented success in various machine learning applications. Formal understanding on what makes these networks so successful is gradually unfolding, but for the most part there are still significant mysteries to unravel. The inductive bias, which reflects prior knowledge embedded in the network architecture, is one of them. In this work, we establish a fundamental connection between the fields of quantum physics and deep learning. We use this connection for asserting novel theoretical observations regarding the role that the number of channels in each layer of the convolutional network fulfills in the overall inductive bias. Specifically, we show an equivalence between the function realized by a deep convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which relies on their common underlying tensorial structure. This facilitates the use of quantum entanglement measures as well-defined quantifiers of a deep network's expressive ability to model intricate correlation structures of its inputs. Most importantly, the construction of a deep ConvAC in terms of a Tensor Network is made available. This description enables us to carry a graph-theoretic analysis of a convolutional network, with which we demonstrate a direct control over the inductive bias of the deep network via its channel numbers, that are related to the min-cut in the underlying graph. This result is relevant to any practitioner designing a network for a specific task. We theoretically analyze ConvACs, and empirically validate our findings on more common ConvNets which involve ReLU activations and max pooling. Beyond the results described above, the description of a deep convolutional network in well-defined graph-theoretic tools and the formal connection to quantum entanglement, are two interdisciplinary bridges that are brought forth by this work.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Convolutional Networks on Graph-Structured Data
• Quantum Entanglement in Neural Network States
• Accelerate Monte Carlo Simulations with Restricted Boltzmann Machines
• Equivalence of restricted Boltzmann machines and tensor network states
• An exact mapping between the Variational Renormalization Group and Deep Learning
• Deep learning and the renormalization group

Cited by in corpus (15)

• The Born Supremacy: Quantum Advantage and Training of an Ising Born Machine
• Machine learning \& artificial intelligence in the quantum domain
• Entanglement-guided architectures of machine learning by quantum tensor network
• Opening the black box of deep learning
• Hybrid quantum-classical classifier based on tensor network and variational quantum circuit
• Analysis and Design of Convolutional Networks via Hierarchical Tensor Decompositions
• Tensor network approaches for learning non-linear dynamical laws
• Machine learning algorithms based on generalized Gibbs ensembles
• A Multi-Scale Tensor Network Architecture for Classification and Regression
• Quantum Deformed Neural Networks
• Provably efficient neural network representation for image classification
• Complexity for deep neural networks and other characteristics of deep feature representations
• Quantum State Assignment Flows
• Tensor Network for Supervised Learning at Finite Temperature
• Self-regularizing restricted Boltzmann machines