Quantum Language Model with Entanglement Embedding for Question Answering

Abstract

Quantum Language Models (QLMs) in which words are modelled as quantum superposition of sememes have demonstrated a high level of model transparency and good post-hoc interpretability. Nevertheless, in the current literature word sequences are basically modelled as a classical mixture of word states, which cannot fully exploit the potential of a quantum probabilistic description. A full quantum model is yet to be developed to explicitly capture the non-classical correlations within the word sequences. We propose a neural network model with a novel Entanglement Embedding (EE) module, whose function is to transform the word sequences into entangled pure states of many-body quantum systems. Strong quantum entanglement, which is the central concept of quantum information and an indication of parallelized correlations among the words, is observed within the word sequences. Numerical experiments show that the proposed QLM with EE (QLM-EE) achieves superior performance compared with the classical deep neural network models and other QLMs on Question Answering (QA) datasets. In addition, the post-hoc interpretability of the model can be improved by quantizing the degree of entanglement among the words.

References in corpus (11)

• Sequence to Sequence Learning with Neural Networks
• Towards A Rigorous Science of Interpretable Machine Learning
• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• Quantum-enhanced machine learning
• Deep Learning for Answer Sentence Selection
• On the Dimensionality of Word Embedding
• Quantum Algorithms for Compositional Natural Language Processing
• Encoding word order in complex embeddings
• A Study of Entanglement in a Categorical Framework of Natural Language
• CNM: An Interpretable Complex-valued Network for Matching
• On the learnability of quantum neural networks

Cited by in corpus (6)

• QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computer
• Residual Tensor Train: A Quantum-inspired Approach for Learning Multiple Multilinear Correlations
• Quantum-machine-assisted Drug Discovery
• lambeq: An Efficient High-Level Python Library for Quantum NLP
• Language Modeling with Reduced Densities
• Quantum Adjoint Convolutional Layers for Effective Data Representation