Revisiting Graph Neural Networks: All We Have is Low-Pass Filters

Abstract

Graph neural networks have become one of the most important techniques to solve machine learning problems on graph-structured data. Recent work on vertex classification proposed deep and distributed learning models to achieve high performance and scalability. However, we find that the feature vectors of benchmark datasets are already quite informative for the classification task, and the graph structure only provides a means to denoise the data. In this paper, we develop a theoretical framework based on graph signal processing for analyzing graph neural networks. Our results indicate that graph neural networks only perform low-pass filtering on feature vectors and do not have the non-linear manifold learning property. We further investigate their resilience to feature noise and propose some insights on GCN-based graph neural network design.

12 pages, 5 figures, 2 tables

References in corpus (2)

• Simplifying Graph Convolutional Networks
• Predicting multicellular function through multi-layer tissue networks

Cited by in corpus (38)

• EDITS: Modeling and Mitigating Data Bias for Graph Neural Networks
• Towards Deeper Graph Neural Networks with Differentiable Group Normalization
• Self-supervised Consensus Representation Learning for Attributed Graph
• Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification?
• Model Degradation Hinders Deep Graph Neural Networks
• Correlating sparse sensing for large-scale traffic speed estimation: A Laplacian-enhanced low-rank tensor kriging approach
• Personalized Graph Signal Processing for Collaborative Filtering
• Robust Mid-Pass Filtering Graph Convolutional Networks
• Dirichlet Energy Constrained Learning for Deep Graph Neural Networks
• Understanding over-squashing and bottlenecks on graphs via curvature
• Graph Neural Networks: Methods, Applications, and Opportunities
• Interpreting and Unifying Graph Neural Networks with An Optimization Framework
• Scalable Graph Neural Networks for Heterogeneous Graphs
• Coupled Layer-wise Graph Convolution for Transportation Demand Prediction
• Graph Representation Learning Beyond Node and Homophily
• Evaluating Deep Graph Neural Networks
• Geometry-enhanced graph neural network for learning the smoothness of glassy dynamics from static structure
• Beltrami Flow and Neural Diffusion on Graphs
• Understanding Graph Neural Networks from Graph Signal Denoising Perspectives
• Dirichlet Graph Variational Autoencoder
• On Graph Neural Networks versus Graph-Augmented MLPs
• Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods
• Network In Graph Neural Network
• DeSCo: Towards Generalizable and Scalable Deep Subgraph Counting
• $p$-Laplacian Based Graph Neural Networks
• Maximum Entropy Weighted Independent Set Pooling for Graph Neural Networks
• Message Passing in Graph Convolution Networks via Adaptive Filter Banks
• Stacked Graph Filter
• SoGCN: Second-Order Graph Convolutional Networks
• AViTMP: A Tracking-Specific Transformer for Single-Branch Visual Tracking
• Graph Denoising with Framelet Regularizer
• Eigen-GNN: A Graph Structure Preserving Plug-in for GNNs
• Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning
• Network representation learning: A macro and micro view
• Hypergraph Neural Sheaf Diffusion: A Symmetric Simplicial Set Framework for Higher-Order Learning
• Learning on Random Balls is Sufficient for Estimating (Some) Graph Parameters
• Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networks
• Efficient Graph based Recommender System with Weighted Averaging of Messages