SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks

Abstract

Graph neural networks (GNNs) work well when the graph structure is provided. However, this structure may not always be available in real-world applications. One solution to this problem is to infer a task-specific latent structure and then apply a GNN to the inferred graph. Unfortunately, the space of possible graph structures grows super-exponentially with the number of nodes and so the task-specific supervision may be insufficient for learning both the structure and the GNN parameters. In this work, we propose the Simultaneous Learning of Adjacency and GNN Parameters with Self-supervision, or SLAPS, a method that provides more supervision for inferring a graph structure through self-supervision. A comprehensive experimental study demonstrates that SLAPS scales to large graphs with hundreds of thousands of nodes and outperforms several models that have been proposed to learn a task-specific graph structure on established benchmarks.

arXiv commentAccepted at NeurIPS 2021

References in corpus (33)

• Adam: A Method for Stochastic Optimization
• Scikit-learn: Machine Learning in Python
• Emergence of scaling in random networks
• Statistical mechanics of complex networks
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Adversarial Attacks on Neural Networks for Graph Data
• Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models
• Few-Shot Learning with Graph Neural Networks
• GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
• Learning Discrete Structures for Graph Neural Networks
• Strategies for Pre-training Graph Neural Networks
• Efficient Graph Generation with Graph Recurrent Attention Networks
• Adversarial Attack on Graph Structured Data
• Graph-Bert: Only Attention is Needed for Learning Graph Representations
• Learning representations of irregular particle-detector geometry with distance-weighted graph networks
• Differentiable Graph Module (DGM) for Graph Convolutional Networks
• Rethinking Attention with Performers
• Self-supervised Learning on Graphs: Deep Insights and New Direction
• Machine Learning on Graphs: A Model and Comprehensive Taxonomy
• Adaptive Graph Convolutional Neural Networks
• A Survey on The Expressive Power of Graph Neural Networks
• When Does Self-Supervision Help Graph Convolutional Networks?
• Graph Structure Learning for Robust Graph Neural Networks
• Deep Iterative and Adaptive Learning for Graph Neural Networks
• Graph Normalizing Flows
• Grale: Designing Networks for Graph Learning
• Self-supervised Training of Graph Convolutional Networks
• Graph Neural Networks with Generated Parameters for Relation Extraction
• Learning Graph Structure With A Finite-State Automaton Layer

Cited by in corpus (5)

• Graph Self-Supervised Learning: A Survey
• Learning Strong Graph Neural Networks with Weak Information
• Collaborate to Adapt: Source-Free Graph Domain Adaptation via Bi-directional Adaptation
• Handling Feature Heterogeneity with Learnable Graph Patches
• Cyclic Graph Attentive Match Encoder (CGAME): A Novel Neural Network For OD Estimation