Spectral Graph Attention Network with Fast Eigen-approximation
arXiv:2003.07450
Abstract
Variants of Graph Neural Networks (GNNs) for representation learning have been proposed recently and achieved fruitful results in various fields. Among them, Graph Attention Network (GAT) first employs a self-attention strategy to learn attention weights for each edge in the spatial domain. However, learning the attentions over edges can only focus on the local information of graphs and greatly increases the computational costs. In this paper, we first introduce the attention mechanism in the spectral domain of graphs and present Spectral Graph Attention Network (SpGAT) that learns representations for different frequency components regarding weighted filters and graph wavelets bases. In this way, SpGAT can better capture global patterns of graphs in an efficient manner with much fewer learned parameters than that of GAT. Further, to reduce the computational cost of SpGAT brought by the eigen-decomposition, we propose a fast approximation variant SpGAT-Cheby. We thoroughly evaluate the performance of SpGAT and SpGAT-Cheby in semi-supervised node classification tasks and verify the effectiveness of the learned attentions in the spectral domain.
arXiv commentAccepted by Deep Learning on Graphs: Method and Applications (DLG-KDD21)
References in corpus (17)
- Adam: A Method for Stochastic Optimization
- The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
- Simplifying Graph Convolutional Networks
- Pitfalls of Graph Neural Network Evaluation
- FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
- Graph Neural Networks with convolutional ARMA filters
- Learning Structural Node Embeddings Via Diffusion Wavelets
- Revisiting Graph Neural Networks: All We Have is Low-Pass Filters
- Adaptive Sampling Towards Fast Graph Representation Learning
- LanczosNet: Multi-Scale Deep Graph Convolutional Networks
- Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
- Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View
- Implicit Graph Neural Networks
- Hypergraph Convolution and Hypergraph Attention
- Multi-hop Attention Graph Neural Network
- Graph-based Isometry Invariant Representation Learning
- AutoGL: A Library for Automated Graph Learning