Improving Expressivity of Graph Neural Networks

Abstract

We propose a Graph Neural Network with greater expressive power than commonly used GNNs - not constrained to only differentiate between graphs that Weisfeiler-Lehman test recognizes to be non-isomorphic. We use a graph attention network with expanding attention window that aggregates information from nodes exponentially far away. We also use partially random initial embeddings, allowing differentiation between nodes that would otherwise look the same. This could cause problem with a traditional dropout mechanism, therefore we use a "head dropout", randomly ignoring some attention heads rather than some dimensions of the embedding.

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Graph Transformer Networks
• How Powerful are Graph Neural Networks?
• Are Powerful Graph Neural Nets Necessary? A Dissection on Graph Classification
• Relational Pooling for Graph Representations
• Graph Star Net for Generalized Multi-Task Learning
• When Work Matters: Transforming Classical Network Structures to Graph CNN