Discriminative Neural Sentence Modeling by Tree-Based Convolution

Abstract

This paper proposes a tree-based convolutional neural network (TBCNN) for discriminative sentence modeling. Our models leverage either constituency trees or dependency trees of sentences. The tree-based convolution process extracts sentences' structural features, and these features are aggregated by max pooling. Such architecture allows short propagation paths between the output layer and underlying feature detectors, which enables effective structural feature learning and extraction. We evaluate our models on two tasks: sentiment analysis and question classification. In both experiments, TBCNN outperforms previous state-of-the-art results, including existing neural networks and dedicated feature/rule engineering. We also make efforts to visualize the tree-based convolution process, shedding light on how our models work.

References in corpus (7)

• Natural Language Processing (almost) from Scratch
• Convolutional Neural Networks for Sentence Classification
• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• A Convolutional Neural Network for Modelling Sentences
• Long Short-Term Memory Over Tree Structures
• Compositional Distributional Semantics with Long Short Term Memory
• Building Program Vector Representations for Deep Learning

Cited by in corpus (21)

• Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling
• Convolutional Neural Networks over Tree Structures for Programming Language Processing
• Improved Relation Classification by Deep Recurrent Neural Networks with Data Augmentation
• Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path
• How Transferable are Neural Networks in NLP Applications?
• Natural Language Inference by Tree-Based Convolution and Heuristic Matching
• Backward and Forward Language Modeling for Constrained Sentence Generation
• Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling
• MGNC-CNN: A Simple Approach to Exploiting Multiple Word Embeddings for Sentence Classification
• StalemateBreaker: A Proactive Content-Introducing Approach to Automatic Human-Computer Conversation
• A Bi-LSTM-RNN Model for Relation Classification Using Low-Cost Sequence Features
• Neural Attribute Machines for Program Generation
• Multichannel CNN with Attention for Text Classification
• Distilling Word Embeddings: An Encoding Approach
• TreeCaps: Tree-Based Capsule Networks for Source Code Processing
• Learning Semantically Coherent and Reusable Kernels in Convolution Neural Nets for Sentence Classification
• AlignVE: Visual Entailment Recognition Based on Alignment Relations
• Refining Raw Sentence Representations for Textual Entailment Recognition via Attention
• Text Classification based on Multi-granularity Attention Hybrid Neural Network
• Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations
• Learning to Compose over Tree Structures via POS Tags