AdaGNN: Graph Neural Networks with Adaptive Frequency Response Filter
arXiv:2104.12840 · doi:10.1145/3459637.3482226
Abstract
Graph Neural Networks have recently become a prevailing paradigm for various high-impact graph analytical problems. Existing efforts can be mainly categorized as spectral-based and spatial-based methods. The major challenge for the former is to find an appropriate graph filter to distill discriminative information from input signals for learning. Recently, myriads of explorations are made to achieve better graph filters, e.g., Graph Convolutional Network (GCN), which leverages Chebyshev polynomial truncation to seek an approximation of graph filters and bridge these two families of methods. Nevertheless, it has been shown in recent studies that GCN and its variants are essentially employing fixed low-pass filters to perform information denoising. Thus their learning capability is rather limited and may over-smooth node representations at deeper layers. To tackle these problems, we develop a novel graph neural network framework AdaGNN with a well-designed adaptive frequency response filter. At its core, AdaGNN leverages a simple but elegant trainable filter that spans across multiple layers to capture the varying importance of different frequency components for node representation learning. The inherent differences among different feature channels are also well captured by the filter. As such, it empowers AdaGNN with stronger expressiveness and naturally alleviates the over-smoothing problem. We empirically validate the effectiveness of the proposed framework on various benchmark datasets. Theoretical analysis is also provided to show the superiority of the proposed AdaGNN. The open-source implementation of AdaGNN can be found here: https://github.com/yushundong/AdaGNN.
arXiv commentPublished as a conference paper at CIKM 2021
References in corpus (26)
- Adam: A Method for Stochastic Optimization
- A Comprehensive Survey on Graph Neural Networks
- The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
- Inductive Representation Learning on Large Graphs
- Neural Message Passing for Quantum Chemistry
- Character-level Convolutional Networks for Text Classification
- Neural Graph Collaborative Filtering
- Spectral Networks and Locally Connected Networks on Graphs
- Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
- Simplifying Graph Convolutional Networks
- Hierarchical Graph Representation Learning with Differentiable Pooling
- Learning Convolutional Neural Networks for Graphs
- DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
- Towards Deeper Graph Neural Networks
- Diffusion-Convolutional Neural Networks
- Simple and Deep Graph Convolutional Networks
- Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning
- Revisiting Graph Neural Networks: All We Have is Low-Pass Filters
- PairNorm: Tackling Oversmoothing in GNNs
- Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
- EDITS: Modeling and Mitigating Data Bias for Graph Neural Networks
- Adaptive Graph Convolutional Neural Networks
- A Note on Over-Smoothing for Graph Neural Networks
- Graph U-Nets
- Recursive Graphical Neural Networks for Text Classification
- Encoding Robust Representation for Graph Generation