Tensor network language model

Abstract

We propose a new statistical model suitable for machine learning of systems with long distance correlations such as natural languages. The model is based on directed acyclic graph decorated by multi-linear tensor maps in the vertices and vector spaces in the edges, called tensor network. Such tensor networks have been previously employed for effective numerical computation of the renormalization group flow on the space of effective quantum field theories and lattice models of statistical mechanics. We provide explicit algebro-geometric analysis of the parameter moduli space for tree graphs, discuss model properties and applications such as statistical translation.

21 pages

References in corpus (4)

• Accelerate Monte Carlo Simulations with Restricted Boltzmann Machines
• An exact mapping between the Variational Renormalization Group and Deep Learning
• On the origin of long-range correlations in texts
• Practical variational tomography for critical 1D systems

Cited by in corpus (6)

• lambeq: An Efficient High-Level Python Library for Quantum NLP
• Shortcut Matrix Product States and its applications
• Tensor network to learn the wavefunction of data
• Language Modeling with Reduced Densities
• Probabilistic Graphical Models and Tensor Networks: A Hybrid Framework
• At the Interface of Algebra and Statistics