Recursive Graphical Neural Networks for Text Classification
arXiv:1909.08166
Abstract
The complicated syntax structure of natural language is hard to be explicitly modeled by sequence-based models. Graph is a natural structure to describe the complicated relation between tokens. The recent advance in Graph Neural Networks (GNN) provides a powerful tool to model graph structure data, but simple graph models such as Graph Convolutional Networks (GCN) suffer from over-smoothing problem, that is, when stacking multiple layers, all nodes will converge to the same value. In this paper, we propose a novel Recursive Graphical Neural Networks model (ReGNN) to represent text organized in the form of graph. In our proposed model, LSTM is used to dynamically decide which part of the aggregated neighbor information should be transmitted to upper layers thus alleviating the over-smoothing problem. Furthermore, to encourage the exchange between the local and global information, a global graph-level node is designed. We conduct experiments on both single and multiple label text classification tasks. Experiment results show that our ReGNN model surpasses the strong baselines significantly in most of the datasets and greatly alleviates the over-smoothing problem.
References in corpus (12)
- Semi-Supervised Classification with Graph Convolutional Networks
- Inductive Representation Learning on Large Graphs
- Graph Neural Networks: A Review of Methods and Applications
- Convolutional Neural Networks for Sentence Classification
- Recurrent Neural Network for Text Classification with Multi-Task Learning
- Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
- Gated Graph Sequence Neural Networks
- Get To The Point: Summarization with Pointer-Generator Networks
- Graph Convolutional Networks for Text Classification
- Joint Embedding of Words and Labels for Text Classification
- Sentence-State LSTM for Text Representation
- A Graph-to-Sequence Model for AMR-to-Text Generation