Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning

Abstract

Graph convolutional network (GCN) has become popular in various natural language processing (NLP) tasks with its superiority in long-term and non-consecutive word interactions. However, existing single-hop graph reasoning in GCN may miss some important non-consecutive dependencies. In this study, we define the spectral graph convolutional network with the high-order dynamic Chebyshev approximation (HDGCN), which augments the multi-hop graph reasoning by fusing messages aggregated from direct and long-term dependencies into one convolutional layer. To alleviate the over-smoothing in high-order Chebyshev approximation, a multi-vote-based cross-attention (MVCAttn) with linear computation complexity is also proposed. The empirical results on four transductive and inductive NLP tasks and the ablation study verify the efficacy of the proposed model. Our source code is available at https://github.com/MathIsAll/HDGCN-pytorch.

References in corpus (8)

• Graph Transformer Networks
• Deep Graph Contrastive Representation Learning
• Simple and Deep Graph Convolutional Networks
• A Generalization of Transformer Networks to Graphs
• Revisiting Graph Neural Networks: All We Have is Low-Pass Filters
• Graph-Bert: Only Attention is Needed for Learning Graph Representations
• GResNet: Graph Residual Network for Reviving Deep GNNs from Suspended Animation
• Get Rid of Suspended Animation Problem: Deep Diffusive Neural Network on Graph Semi-Supervised Classification