Graph Neural Networks with Learnable Structural and Positional Representations
arXiv:2110.07875
Abstract
Graph neural networks (GNNs) have become the standard learning architectures for graphs. GNNs have been applied to numerous domains ranging from quantum chemistry, recommender systems to knowledge graphs and natural language processing. A major issue with arbitrary graphs is the absence of canonical positional information of nodes, which decreases the representation power of GNNs to distinguish e.g. isomorphic nodes and other graph symmetries. An approach to tackle this issue is to introduce Positional Encoding (PE) of nodes, and inject it into the input layer, like in Transformers. Possible graph PE are Laplacian eigenvectors. In this work, we propose to decouple structural and positional representations to make easy for the network to learn these two essential properties. We introduce a novel generic architecture which we call LSPE (Learnable Structural and Positional Encodings). We investigate several sparse and fully-connected (Transformer-like) GNNs, and observe a performance increase for molecular datasets, from 1.79% up to 64.14% when considering learnable PE for both GNN classes.
arXiv commentCode at https://github.com/vijaydwivedi75/gnn-lspe
References in corpus (22)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Neural Message Passing for Quantum Chemistry
- Junction Tree Variational Autoencoder for Molecular Graph Generation
- Open Graph Benchmark: Datasets for Machine Learning on Graphs
- A Generalization of Transformer Networks to Graphs
- Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
- DeeperGCN: All You Need to Train Deeper GCNs
- Residual Gated Graph ConvNets
- Benchmarking Graph Neural Networks
- Fake News Detection on Social Media using Geometric Deep Learning
- Do Transformers Really Perform Bad for Graph Representation?
- Position-aware Graph Neural Networks
- On the Bottleneck of Graph Neural Networks and its Practical Implications
- A Survey on The Expressive Power of Graph Neural Networks
- Rethinking Graph Transformers with Spectral Attention
- GraphiT: Encoding Graph Structure in Transformers
- Learning Symbolic Physics with Graph Networks
- On the Equivalence between Positional Node Embeddings and Structural Graph Representations
- Expressive Power of Invariant and Equivariant Graph Neural Networks
- On Positional and Structural Node Features for Graph Neural Networks on Non-attributed Graphs
- Weisfeiler and Lehman Go Cellular: CW Networks