Dependency Sensitive Convolutional Neural Networks for Modeling Sentences and Documents
arXiv:1611.02361
Abstract
The goal of sentence and document modeling is to accurately represent the meaning of sentences and documents for various Natural Language Processing tasks. In this work, we present Dependency Sensitive Convolutional Neural Networks (DSCNN) as a general-purpose classification system for both sentences and documents. DSCNN hierarchically builds textual representations by processing pretrained word embeddings via Long Short-Term Memory networks and subsequently extracting features with convolution operators. Compared with existing recursive neural models with tree structures, DSCNN does not rely on parsers and expensive phrase labeling, and thus is not restricted to sentence-level tasks. Moreover, unlike other CNN-based models that analyze sentences locally by sliding windows, our system captures both the dependency information within each sentence and relationships across sentences in the same document. Experiment results demonstrate that our approach is achieving state-of-the-art performance on several tasks, including sentiment analysis, question type classification, and subjectivity classification.
NAACL2016
References in corpus (5)
- Improving neural networks by preventing co-adaptation of feature detectors
- ADADELTA: An Adaptive Learning Rate Method
- Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
- A Hierarchical Neural Autoencoder for Paragraphs and Documents
- Dependency-based Convolutional Neural Networks for Sentence Embedding
Cited by in corpus (6)
- Densely Connected Bidirectional LSTM with Applications to Sentence Classification
- Cross-lingual Distillation for Text Classification
- An Intelligent CNN-VAE Text Representation Technology Based on Text Semantics for Comprehensive Big Data
- Hierarchical Neural Networks for Sequential Sentence Classification in Medical Scientific Abstracts
- Translations as Additional Contexts for Sentence Classification
- Self-Balanced Dropout