Quantum Language Model with Entanglement Embedding for Question Answering
arXiv:2008.09943 · doi:10.1109/TCYB.2021.3131252
Abstract
Quantum Language Models (QLMs) in which words are modelled as quantum superposition of sememes have demonstrated a high level of model transparency and good post-hoc interpretability. Nevertheless, in the current literature word sequences are basically modelled as a classical mixture of word states, which cannot fully exploit the potential of a quantum probabilistic description. A full quantum model is yet to be developed to explicitly capture the non-classical correlations within the word sequences. We propose a neural network model with a novel Entanglement Embedding (EE) module, whose function is to transform the word sequences into entangled pure states of many-body quantum systems. Strong quantum entanglement, which is the central concept of quantum information and an indication of parallelized correlations among the words, is observed within the word sequences. Numerical experiments show that the proposed QLM with EE (QLM-EE) achieves superior performance compared with the classical deep neural network models and other QLMs on Question Answering (QA) datasets. In addition, the post-hoc interpretability of the model can be improved by quantizing the degree of entanglement among the words.
References in corpus (28)
- Efficient Estimation of Word Representations in Vector Space
- Distributed Representations of Words and Phrases and their Compositionality
- Sequence to Sequence Learning with Neural Networks
- Quantum entanglement
- Quantum Machine Learning
- Towards A Rigorous Science of Interpretable Machine Learning
- Supervised learning with quantum enhanced feature spaces
- Barren plateaus in quantum neural network training landscapes
- Convolutional Neural Network Architectures for Matching Natural Language Sentences
- Quantum-enhanced machine learning
- LSTM-based Deep Learning Models for Non-factoid Answer Selection
- Deep Learning for Answer Sentence Selection
- Attentive Pooling Networks
- Neural Variational Inference for Text Processing
- Adaptive Bayesian Quantum Tomography
- Fidelity-based Probabilistic Q-learning for Control of Quantum Systems
- Recursively Adaptive Quantum State Tomography: Theory and Two-qubit Experiment
- On the Dimensionality of Word Embedding
- Learning-based Quantum Robust Control: Algorithm, Applications and Experiments
- Quantum Algorithms for Compositional Natural Language Processing
- Encoding word order in complex embeddings
- A Study of Entanglement in a Categorical Framework of Natural Language
- Benchmarking neural networks for quantum computation
- aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model
- Quantum Ensemble Classification: A Sampling-based Learning Control Approach
- A Quantum Many-body Wave Function Inspired Language Modeling Approach
- CNM: An Interpretable Complex-valued Network for Matching
- On the learnability of quantum neural networks
Cited by in corpus (6)
- QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computer
- Residual Tensor Train: A Quantum-inspired Approach for Learning Multiple Multilinear Correlations
- Quantum-machine-assisted Drug Discovery
- lambeq: An Efficient High-Level Python Library for Quantum NLP
- Language Modeling with Reduced Densities
- Quantum Adjoint Convolutional Layers for Effective Data Representation