A Convolutional Neural Network for Modelling Sentences

Abstract

The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline.

References in corpus (3)

• Improving neural networks by preventing co-adaptation of feature detectors
• Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars
• Mathematical Foundations for a Compositional Distributional Model of Meaning

Cited by in corpus (10)

• Convolutional Neural Networks for Sentence Classification
• Deep Learning for Answer Sentence Selection
• Evaluating Two-Stream CNN for Video Classification
• Discriminative Neural Sentence Modeling by Tree-Based Convolution
• EmoNets: Multimodal deep learning approaches for emotion recognition in video
• Topic2Vec: Learning Distributed Representations of Topics
• Compositional Distributional Semantics with Long Short Term Memory
• Distributed Representations for Compositional Semantics
• Context-Dependent Translation Selection Using Convolutional Neural Network
• Joint Learning of Distributed Representations for Images and Texts