SIGN: Scalable Inception Graph Neural Networks

Abstract

Graph representation learning has recently been applied to a broad spectrum of problems ranging from computer graphics and chemistry to high energy physics and social media. The popularity of graph neural networks has sparked interest, both in academia and in industry, in developing methods that scale to very large graphs such as Facebook or Twitter social networks. In most of these approaches, the computational cost is alleviated by a sampling strategy retaining a subset of node neighbors or subgraphs at training time. In this paper we propose a new, efficient and scalable graph deep learning architecture which sidesteps the need for graph sampling by using graph convolutional filters of different size that are amenable to efficient precomputation, allowing extremely fast training and inference. Our architecture allows using different local graph operators (e.g. motif-induced adjacency matrices or Personalized Page Rank diffusion matrix) to best suit the task at hand. We conduct extensive experimental evaluation on various open benchmarks and show that our approach is competitive with other state-of-the-art architectures, while requiring a fraction of the training and inference time. Moreover, we obtain state-of-the-art results on ogbn-papers100M, the largest public graph dataset, with over 110 million nodes and 1.5 billion edges.

arXiv commentExtended experiments to ogbn-papers100M

References in corpus (26)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Geometric deep learning: going beyond Euclidean data
• Modeling polypharmacy side effects with graph convolutional networks
• Deep Convolutional Networks on Graph-Structured Data
• Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
• Simplifying Graph Convolutional Networks
• Representation Learning on Graphs with Jumping Knowledge Networks
• Convolutional Networks on Graphs for Learning Molecular Fingerprints
• Disease Prediction using Graph Convolutional Networks: Application to Autism Spectrum Disorder and Alzheimer's Disease
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• Pitfalls of Graph Neural Network Evaluation
• Predicting multicellular function through multi-layer tissue networks
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Graph Neural Networks with convolutional ARMA filters
• GraphSAINT: Graph Sampling Based Inductive Learning Method
• Revisiting Graph Neural Networks: All We Have is Low-Pass Filters
• Topological Signal Processing over Simplicial Complexes
• PairNorm: Tackling Oversmoothing in GNNs
• Adaptive Sampling Towards Fast Graph Representation Learning
• Fake News Detection on Social Media using Geometric Deep Learning
• Deep Learning on Graphs: A Survey
• On the Bottleneck of Graph Neural Networks and its Practical Implications
• Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks
• Geometric deep learning on graphs and manifolds using mixture model CNNs

Cited by in corpus (59)

• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• Benchmarking Graph Neural Networks
• Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
• ETA Prediction with Graph Neural Networks in Google Maps
• A Survey on Graph-Based Deep Learning for Computational Histopathology
• OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs
• Combining Label Propagation and Simple Models Out-performs Graph Neural Networks
• RA-GCN: Graph Convolutional Network for Disease Prediction Problems with Imbalanced Data
• Graph Attention Multi-Layer Perceptron
• Temporal Graph Networks for Deep Learning on Dynamic Graphs
• On Representation Knowledge Distillation for Graph Neural Networks
• Accelerating Large Scale Real-Time GNN Inference using Channel Pruning
• Label Information Enhanced Fraud Detection against Low Homophily in Graphs
• PaSca: a Graph Neural Architecture Search System under the Scalable Paradigm
• Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation
• Model Degradation Hinders Deep Graph Neural Networks
• The Expressive Power of Graph Neural Networks: A Survey
• Understanding over-squashing and bottlenecks on graphs via curvature
• Efficient Heterogeneous Graph Learning via Random Projection
• Bag of Tricks for Node Classification with Graph Neural Networks
• Scalable Graph Neural Networks for Heterogeneous Graphs
• Lifelong Learning of Graph Neural Networks for Open-World Node Classification
• ROD: Reception-aware Online Distillation for Sparse Graphs
• Lifelong Learning on Evolving Graphs Under the Constraints of Imbalanced Classes and New Classes
• Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced training
• Improving Graph Neural Networks with Simple Architecture Design
• Dissecting the Diffusion Process in Linear Graph Convolutional Networks
• Evaluating Deep Graph Neural Networks
• Residual Network and Embedding Usage: New Tricks of Node Classification with Graph Convolutional Networks
• Beltrami Flow and Neural Diffusion on Graphs
• Large-scale graph representation learning with very deep GNNs and self-supervision
• Optimal Solving of Constrained Path-Planning Problems with Graph Convolutional Networks and Optimized Tree Search
• Counterfactual Learning on Graphs: A Survey
• Node Dependent Local Smoothing for Scalable Graph Learning
• VQ-GNN: A Universal Framework to Scale up Graph Neural Networks using Vector Quantization
• On the Equivalence of Decoupled Graph Convolution Network and Label Propagation
• On Graph Neural Networks versus Graph-Augmented MLPs
• How Framelets Enhance Graph Neural Networks
• Graph Neural Networks with Parallel Neighborhood Aggregations for Graph Classification
• Multi-Graph based Multi-Scenario Recommendation in Large-scale Online Video Services
• Utilising Graph Machine Learning within Drug Discovery and Development
• Modeling multi-scale data via a network of networks
• Scalable Multitask Learning Using Gradient-based Estimation of Task Affinity
• GMLP: Building Scalable and Flexible Graph Neural Networks with Feature-Message Passing
• Edge Proposal Sets for Link Prediction
• Why Propagate Alone? Parallel Use of Labels and Features on Graphs
• Boosting Multitask Learning on Graphs through Higher-Order Task Affinities
• Search For Deep Graph Neural Networks
• Local2Global: Scaling global representation learning on graphs via local training
• Graph Neural Networks with Feature and Structure Aware Random Walk
• LG4AV: Combining Language Models and Graph Neural Networks for Author Verification
• Structure-Aware Label Smoothing for Graph Neural Networks
• Focus Where It Matters: Graph Selective State Focused Attention Networks
• Analyzing the Performance of Graph Neural Networks with Pipe Parallelism
• Sequential Aggregation and Rematerialization: Distributed Full-batch Training of Graph Neural Networks on Large Graphs
• SAS: A Simple, Accurate and Scalable Node Classification Algorithm
• Efficient Variational Graph Autoencoders for Unsupervised Cross-domain Prerequisite Chains
• Scaling Up Graph Neural Networks Via Graph Coarsening
• Fast Graph Learning with Unique Optimal Solutions