Attention-based Graph Neural Network for Semi-supervised Learning

Abstract

Recently popularized graph neural networks achieve the state-of-the-art accuracy on a number of standard benchmark datasets for graph-based semi-supervised learning, improving significantly over existing approaches. These architectures alternate between a propagation layer that aggregates the hidden states of the local neighborhood and a fully-connected layer. Perhaps surprisingly, we show that a linear model, that removes all the intermediate fully-connected layers, is still able to achieve a performance comparable to the state-of-the-art models. This significantly reduces the number of parameters, which is critical for semi-supervised learning where number of labeled examples are small. This in turn allows a room for designing more innovative propagation layers. Based on this insight, we propose a novel graph neural network that removes all the intermediate fully-connected layers, and replaces the propagation layers with attention mechanisms that respect the structure of the graph. The attention mechanism allows us to learn a dynamic and adaptive local summary of the neighborhood to achieve more accurate predictions. In a number of experiments on benchmark citation networks datasets, we demonstrate that our approach outperforms competing methods. By examining the attention weights among neighbors, we show that our model provides some interesting insights on how neighbors influence each other.

References in corpus (12)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Semi-Supervised Classification with Graph Convolutional Networks
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Geometric deep learning: going beyond Euclidean data
• Deep Convolutional Networks on Graph-Structured Data
• Graph Convolutional Matrix Completion
• Learning Combinatorial Optimization Algorithms over Graphs
• Learning Convolutional Neural Networks for Graphs
• Discriminative Embeddings of Latent Variable Models for Structured Data
• Robust Spatial Filtering with Graph Convolutional Neural Networks

Cited by in corpus (48)

• A General Survey on Attention Mechanisms in Deep Learning
• Causal Attention for Interpretable and Generalizable Graph Classification
• Graph Neural Networks: Taxonomy, Advances and Trends
• Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentation
• How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision
• Unifying Graph Convolutional Neural Networks and Label Propagation
• Learning Multi-Attention Context Graph for Group-Based Re-Identification
• Directed Graph Convolutional Network
• Scalable Graph Neural Networks via Bidirectional Propagation
• Contrastive Graph Pooling for Explainable Classification of Brain Networks
• How does spatial structure affect psychological restoration? A method based on Graph Neural Networks and Street View Imagery
• How Attentive are Graph Attention Networks?
• Multi-hop Attention Graph Neural Network
• A Spatial-Temporal Graph Neural Network Framework for Automated Software Bug Triaging
• Semantic and Relational Spaces in Science of Science: Deep Learning Models for Article Vectorisation
• Understanding the Message Passing in Graph Neural Networks via Power Iteration Clustering
• HAHE: Hierarchical Attentive Heterogeneous Information Network Embedding
• Improving Graph Attention Networks with Large Margin-based Constraints
• Boost then Convolve: Gradient Boosting Meets Graph Neural Networks
• Label Efficient Semi-Supervised Learning via Graph Filtering
• Graph Inference Learning for Semi-supervised Classification
• PAN: Path Integral Based Convolution for Deep Graph Neural Networks
• SimGNN: A Neural Network Approach to Fast Graph Similarity Computation
• Fast Graph Attention Networks Using Effective Resistance Based Graph Sparsification
• Understanding Graph Neural Networks from Graph Signal Denoising Perspectives
• FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systems
• FedGL: Federated Graph Learning Framework with Global Self-Supervision
• Graph Convolutional Value Decomposition in Multi-Agent Reinforcement Learning
• Semi-Supervised Node Classification by Graph Convolutional Networks and Extracted Side Information
• Neural Dynamics on Complex Networks
• Human Action Recognition with Multi-Laplacian Graph Convolutional Networks
• $p$-Laplacian Based Graph Neural Networks
• Manifold Graph with Learned Prototypes for Semi-Supervised Image Classification
• PushNet: Efficient and Adaptive Neural Message Passing
• PINE: Universal Deep Embedding for Graph Nodes via Partial Permutation Invariant Set Functions
• Recurrent Attention Walk for Semi-supervised Classification
• Progressive Representative Labeling for Deep Semi-Supervised Learning
• BiTe-GCN: A New GCN Architecture via BidirectionalConvolution of Topology and Features on Text-Rich Networks
• Multiscale Laplacian Learning
• Graph Information Vanishing Phenomenon inImplicit Graph Neural Networks
• Fea2Fea: Exploring Structural Feature Correlations via Graph Neural Networks
• Graphs, Entities, and Step Mixture
• Contrastive Adaptive Propagation Graph Neural Networks for Efficient Graph Learning
• LW-GCN: A Lightweight FPGA-based Graph Convolutional Network Accelerator
• Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node features
• Fast Haar Transforms for Graph Neural Networks
• Network representation learning: A macro and micro view
• Lookup subnet based Spatial Graph Convolutional neural Network