How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision

Abstract

Attention mechanism in graph neural networks is designed to assign larger weights to important neighbor nodes for better representation. However, what graph attention learns is not understood well, particularly when graphs are noisy. In this paper, we propose a self-supervised graph attention network (SuperGAT), an improved graph attention model for noisy graphs. Specifically, we exploit two attention forms compatible with a self-supervised task to predict edges, whose presence and absence contain the inherent information about the importance of the relationships between nodes. By encoding edges, SuperGAT learns more expressive attention in distinguishing mislinked neighbors. We find two graph characteristics influence the effectiveness of attention forms and self-supervision: homophily and average degree. Thus, our recipe provides guidance on which attention design to use when those two graph characteristics are known. Our experiment on 17 real-world datasets demonstrates that our recipe generalizes across 15 datasets of them, and our models designed by recipe show improved performance over baselines.

ICLR 2021

References in corpus (8)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Deep Convolutional Networks on Graph-Structured Data
• Fast Graph Representation Learning with PyTorch Geometric
• Variational Graph Auto-Encoders
• Predicting multicellular function through multi-layer tissue networks
• Geom-GCN: Geometric Graph Convolutional Networks
• Measuring and Improving the Use of Graph Information in Graph Neural Networks
• Improving Graph Attention Networks with Large Margin-based Constraints

Cited by in corpus (24)

• Causal Attention for Interpretable and Generalizable Graph Classification
• A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability
• Variational Graph Normalized Auto-Encoders
• Unnoticeable Backdoor Attacks on Graph Neural Networks
• Learning from the Dark: Boosting Graph Convolutional Neural Networks with Diverse Negative Samples
• Graph Neural Networks: Methods, Applications, and Opportunities
• Exploring Edge Disentanglement for Node Classification
• GraphWorld: Fake Graphs Bring Real Insights for GNNs
• Graph-adaptive Rectified Linear Unit for Graph Neural Networks
• Spatial Heterophily Aware Graph Neural Networks
• Counterfactual Learning on Graphs: A Survey
• Finding Heterophilic Neighbors via Confidence-based Subgraph Matching for Semi-supervised Node Classification
• Simple Truncated SVD based Model for Node Classification on Heterophilic Graphs
• Label-Wise Graph Convolutional Network for Heterophilic Graphs
• Deep Attention-guided Graph Clustering with Dual Self-supervision
• LaSER: Language-Specific Event Recommendation
• Attention-driven Graph Clustering Network
• GLAM: Graph Learning by Modeling Affinity to Labeled Nodes for Graph Neural Networks
• Embedding Graphs on Grassmann Manifold
• Gradient scarcity with Bilevel Optimization for Graph Learning
• Effective Eigendecomposition based Graph Adaptation for Heterophilic Networks
• LSP : Acceleration and Regularization of Graph Neural Networks via Locality Sensitive Pruning of Graphs
• Towards Self-Explainable Graph Neural Network
• CoRGi: Content-Rich Graph Neural Networks with Attention