Graph Neural Networks with Learnable Structural and Positional Representations
arXiv:2110.07875
Abstract
Graph neural networks (GNNs) have become the standard learning architectures for graphs. GNNs have been applied to numerous domains ranging from quantum chemistry, recommender systems to knowledge graphs and natural language processing. A major issue with arbitrary graphs is the absence of canonical positional information of nodes, which decreases the representation power of GNNs to distinguish e.g. isomorphic nodes and other graph symmetries. An approach to tackle this issue is to introduce Positional Encoding (PE) of nodes, and inject it into the input layer, like in Transformers. Possible graph PE are Laplacian eigenvectors. In this work, we propose to decouple structural and positional representations to make easy for the network to learn these two essential properties. We introduce a novel generic architecture which we call LSPE (Learnable Structural and Positional Encodings). We investigate several sparse and fully-connected (Transformer-like) GNNs, and observe a performance increase for molecular datasets, from 1.79% up to 64.14% when considering learnable PE for both GNN classes.
Code at https://github.com/vijaydwivedi75/gnn-lspe
References in corpus (12)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- A Generalization of Transformer Networks to Graphs
- DeeperGCN: All You Need to Train Deeper GCNs
- Fake News Detection on Social Media using Geometric Deep Learning
- Do Transformers Really Perform Bad for Graph Representation?
- Position-aware Graph Neural Networks
- Rethinking Graph Transformers with Spectral Attention
- GraphiT: Encoding Graph Structure in Transformers
- Learning Symbolic Physics with Graph Networks
- On Positional and Structural Node Features for Graph Neural Networks on Non-attributed Graphs
- Weisfeiler and Lehman Go Cellular: CW Networks