Evaluating Neural Word Representations in Tensor-Based Compositional Settings

Abstract

We provide a comparative study between neural word representations and traditional vector spaces based on co-occurrence counts, in a number of compositional tasks. We use three different semantic spaces and implement seven tensor-based compositional models, which we then test (together with simpler additive and multiplicative approaches) in tasks involving verb disambiguation and sentence similarity. To check their scalability, we additionally evaluate the spaces using simple compositional methods on larger-scale tasks with less constrained language: paraphrase detection and dialogue act tagging. In the more constrained tasks, co-occurrence vectors are competitive, although choice of compositional method is important; on the larger-scale tasks, they are outperformed by neural word embeddings, which show robust, stable performance across the tasks.

To be published in EMNLP 2014

References in corpus (7)

• Distributed Representations of Words and Phrases and their Compositionality
• From Frequency to Meaning: Vector Space Models of Semantics
• word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
• Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech
• Experimental Support for a Categorical Compositional Distributional Model of Meaning
• A Study of Entanglement in a Categorical Framework of Natural Language
• Experimenting with Transitive Verbs in a DisCoCat

Cited by in corpus (5)

• Objects2action: Classifying and localizing actions without any video example
• Learning to Understand Phrases by Embedding the Dictionary
• Adaptive Joint Learning of Compositional and Non-Compositional Phrase Embeddings
• Evaluation of Unsupervised Compositional Representations
• A Typedriven Vector Semantics for Ellipsis with Anaphora using Lambek Calculus with Limited Contraction