How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision

Abstract

Attention mechanism in graph neural networks is designed to assign larger weights to important neighbor nodes for better representation. However, what graph attention learns is not understood well, particularly when graphs are noisy. In this paper, we propose a self-supervised graph attention network (SuperGAT), an improved graph attention model for noisy graphs. Specifically, we exploit two attention forms compatible with a self-supervised task to predict edges, whose presence and absence contain the inherent information about the importance of the relationships between nodes. By encoding edges, SuperGAT learns more expressive attention in distinguishing mislinked neighbors. We find two graph characteristics influence the effectiveness of attention forms and self-supervision: homophily and average degree. Thus, our recipe provides guidance on which attention design to use when those two graph characteristics are known. Our experiment on 17 real-world datasets demonstrates that our recipe generalizes across 15 datasets of them, and our models designed by recipe show improved performance over baselines.

ICLR 2021

References in corpus (8)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Deep Convolutional Networks on Graph-Structured Data
• Fast Graph Representation Learning with PyTorch Geometric
• Variational Graph Auto-Encoders
• Predicting multicellular function through multi-layer tissue networks
• Geom-GCN: Geometric Graph Convolutional Networks
• Measuring and Improving the Use of Graph Information in Graph Neural Networks
• Improving Graph Attention Networks with Large Margin-based Constraints

Cited by in corpus (16)

• Variational Graph Normalized Auto-Encoders
• Unnoticeable Backdoor Attacks on Graph Neural Networks
• Learning from the Dark: Boosting Graph Convolutional Neural Networks with Diverse Negative Samples
• Graph Neural Networks: Methods, Applications, and Opportunities
• Exploring Edge Disentanglement for Node Classification
• Graph-adaptive Rectified Linear Unit for Graph Neural Networks
• Finding Heterophilic Neighbors via Confidence-based Subgraph Matching for Semi-supervised Node Classification
• Simple Truncated SVD based Model for Node Classification on Heterophilic Graphs
• LaSER: Language-Specific Event Recommendation
• Attention-driven Graph Clustering Network
• Embedding Graphs on Grassmann Manifold
• GLAM: Graph Learning by Modeling Affinity to Labeled Nodes for Graph Neural Networks
• Gradient scarcity with Bilevel Optimization for Graph Learning
• Effective Eigendecomposition based Graph Adaptation for Heterophilic Networks
• Towards Self-Explainable Graph Neural Network
• CoRGi: Content-Rich Graph Neural Networks with Attention