A Comprehensive Benchmark on Spectral GNNs: The Impact on Efficiency, Memory, and Effectiveness

Abstract

With recent advancements in graph neural networks (GNNs), spectral GNNs have received increasing popularity by virtue of their ability to retrieve graph signals in the spectral domain. These models feature uniqueness in efficient computation as well as rich expressiveness, which stems from advanced management and profound understanding of graph data. However, few systematic studies have been conducted to assess spectral GNNs, particularly in benchmarking their efficiency, memory consumption, and effectiveness in a unified and fair manner. There is also a pressing need to select spectral models suitable for learning specific graph data and deploying them to massive web-scale graphs, which is currently constrained by the varied model designs and training settings. In this work, we extensively benchmark spectral GNNs with a focus on the spectral perspective, demystifying them as spectral graph filters. We analyze and categorize 35 GNNs with 27 corresponding filters, spanning diverse formulations and utilizations of the graph data. Then, we implement the filters within a unified spectral-oriented framework with dedicated graph computations and efficient training schemes. In particular, our implementation enables the deployment of spectral GNNs over million-scale graphs and various tasks with comparable performance and less overhead. Thorough experiments are conducted on the graph filters with comprehensive metrics on effectiveness and efficiency, offering novel observations and practical guidelines that are only available from our evaluations across graph scales. Different from the prevailing belief, our benchmark reveals an intricate landscape regarding the effectiveness and efficiency of spectral graph filters, demonstrating the potential to achieve desirable performance through tailored spectral manipulation of graph data.

Technical Report with Appendix. Our code and evaluation is available at: https://github.com/gdmnl/Spectral-GNN-Benchmark

References in corpus (19)

• A Comprehensive Survey on Graph Neural Networks
• Graph Convolutional Neural Networks for Web-Scale Recommender Systems
• Towards Deeper Graph Neural Networks
• Graph Neural Networks with convolutional ARMA filters
• Graph signal processing for machine learning: A review and new perspectives
• Investigating and Mitigating Degree-Related Biases in Graph Convolutional Networks
• Accelerating Large Scale Real-Time GNN Inference using Channel Pruning
• HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework
• AdaGNN: Graph Neural Networks with Adaptive Frequency Response Filter
• Approximate Graph Propagation
• Ginex: SSD-enabled Billion-scale Graph Neural Network Training on a Single Machine via Provably Optimal In-memory Caching
• Grain: Improving Data Efficiency of Graph Neural Networks via Diversified Influence Maximization
• Hierarchical Graph Transformer with Adaptive Node Sampling
• Algorithm and System Co-design for Efficient Subgraph-based Graph Representation Learning
• SCARA: Scalable Graph Neural Networks with Feature-Oriented Optimization
• A Comprehensive Study on Large-Scale Graph Training: Benchmarking and Rethinking
• Dissecting the Diffusion Process in Linear Graph Convolutional Networks
• Beyond Real-world Benchmark Datasets: An Empirical Study of Node Classification with GNNs
• D3-GNN: Dynamic Distributed Dataflow for Streaming Graph Neural Networks