Should Semantic Vector Composition be Explicit? Can it be Linear?

Abstract

Vector representations have become a central element in semantic language modelling, leading to mathematical overlaps with many fields including quantum theory. Compositionality is a core goal for such representations: given representations for 'wet' and 'fish', how should the concept 'wet fish' be represented? This position paper surveys this question from two points of view. The first considers the question of whether an explicit mathematical representation can be successful using only tools from within linear algebra, or whether other mathematical tools are needed. The second considers whether semantic vector composition should be explicitly described mathematically, or whether it can be a model-internal side-effect of training a neural network. A third and newer question is whether a compositional model can be implemented on a quantum computer. Given the fundamentally linear nature of quantum mechanics, we propose that these questions are related, and that this survey may help to highlight candidate operations for future quantum implementation.

References in corpus (18)

• Efficient Estimation of Word Representations in Vector Space
• word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
• Bifurcation-based adiabatic quantum computation with a nonlinear oscillator network: Toward quantum soft computing
• Vector Symbolic Architectures answer Jackendoff's challenges for cognitive neuroscience
• Quantum Aspects of Semantic Analysis and Symbolic Artificial Intelligence
• QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computer
• Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs
• The Frobenius anatomy of word meanings II: possessive relative pronouns
• Sentence Entailment in Compositional Distributional Semantics
• Compositional Distributional Semantics with Compact Closed Categories and Frobenius Algebras
• Quantum Mathematics in Artificial Intelligence
• Quantum Language Processing
• Towards logical negation for compositional distributional semantics
• Combining pre-trained language models and structured knowledge
• Quantum polar decomposition algorithm
• Uniqueness of Composition in Quantum Theory and Linguistics
• Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees
• At the Interface of Algebra and Statistics