Stochastic Aggregation in Graph Neural Networks

Abstract

Graph neural networks (GNNs) manifest pathologies including over-smoothing and limited discriminating power as a result of suboptimally expressive aggregating mechanisms. We herein present a unifying framework for stochastic aggregation (STAG) in GNNs, where noise is (adaptively) injected into the aggregation process from the neighborhood to form node embeddings. We provide theoretical arguments that STAG models, with little overhead, remedy both of the aforementioned problems. In addition to fixed-noise models, we also propose probabilistic versions of STAG models and a variational inference framework to learn the noise posterior. We conduct illustrative experiments clearly targeting oversmoothing and multiset aggregation limitations. Furthermore, STAG enhances general performance of GNNs demonstrated by competitive performance in common citation and molecule graph benchmark datasets.

References in corpus (4)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Inductive Representation Learning on Large Graphs
• How Powerful are Graph Neural Networks?
• A Note on Over-Smoothing for Graph Neural Networks

Cited by in corpus (4)

• Machine-learned molecular mechanics force field for the simulation of protein-ligand systems and beyond
• On the design space between molecular mechanics and machine learning force fields
• Molecular Dynamics and Machine Learning Unlock Possibilities in Beauty Design -- A Perspective
• Adaptive Neural Message Passing for Inductive Learning on Hypergraphs