Dual-Attention Graph Convolutional Network
arXiv:1911.12486
Abstract
Graph convolutional networks (GCNs) have shown the powerful ability in text structure representation and effectively facilitate the task of text classification. However, challenges still exist in adapting GCN on learning discriminative features from texts due to the main issue of graph variants incurred by the textual complexity and diversity. In this paper, we propose a dual-attention GCN to model the structural information of various texts as well as tackle the graph-invariant problem through embedding two types of attention mechanisms, i.e. the connection-attention and hop-attention, into the classic GCN. To encode various connection patterns between neighbour words, connection-attention adaptively imposes different weights specified to neighbourhoods of each word, which captures the short-term dependencies. On the other hand, the hop-attention applies scaled coefficients to different scopes during the graph diffusion process to make the model learn more about the distribution of context, which captures long-term semantics in an adaptive way. Extensive experiments are conducted on five widely used datasets to evaluate our dual-attention GCN, and the achieved state-of-the-art performance verifies the effectiveness of dual-attention mechanisms.
References in corpus (10)
- Distributed Representations of Words and Phrases and their Compositionality
- Semi-Supervised Classification with Graph Convolutional Networks
- Distributed Representations of Sentences and Documents
- Inductive Representation Learning on Large Graphs
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- Deep Convolutional Networks on Graph-Structured Data
- Recurrent Neural Network for Text Classification with Multi-Task Learning
- Recurrent Models of Visual Attention
- Graph Convolutional Networks for Text Classification
- Sentence-State LSTM for Text Representation