Compositionality for Recursive Neural Networks

Abstract

Modelling compositionality has been a longstanding area of research in the field of vector space semantics. The categorical approach to compositionality maps grammar onto vector spaces in a principled way, but comes under fire for requiring the formation of very high-dimensional matrices and tensors, and therefore being computationally infeasible. In this paper I show how a linear simplification of recursive neural tensor network models can be mapped directly onto the categorical approach, giving a way of computing the required matrices and tensors. This mapping suggests a number of lines of research for both categorical compositional vector space models of meaning and for recursive neural network models of compositionality.

arXiv commentpresented at NeSy2018, Thirteenth International Workshop on Neural-Symbolic Learning and Reasoning, co-located with Human-Level AI 2018, Prague, CZ, August 23-24, 2018

References in corpus (2)

• The Frobenius anatomy of word meanings I: subject and object relative pronouns
• Multi-Step Regression Learning for Compositional Distributional Semantics

Cited by in corpus (1)

• Lambek pregroups are Frobenius spiders in preorders