Learning to Pool in Graph Neural Networks for Extrapolation

Abstract

Graph neural networks (GNNs) are one of the most popular approaches to using deep learning on graph-structured data, and they have shown state-of-the-art performances on a variety of tasks. However, according to a recent study, a careful choice of pooling functions, which are used for the aggregation and readout operations in GNNs, is crucial for enabling GNNs to extrapolate. Without proper choices of pooling functions, which varies across tasks, GNNs completely fail to generalize to out-of-distribution data, while the number of possible choices grows exponentially with the number of layers. In this paper, we present GNP, a $L^p$ norm-like pooling function that is trainable end-to-end for any given task. Notably, GNP generalizes most of the widely-used pooling functions. We verify experimentally that simply using GNP for every aggregation and readout operation enables GNNs to extrapolate well on many node-level, graph-level, and set-related tasks; and GNP sometimes performs even better than the best-performing choices among existing pooling functions.

References in corpus (9)

• Emergence of scaling in random networks
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Fast Graph Representation Learning with PyTorch Geometric
• Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks
• TUDataset: A collection of benchmark datasets for learning with graphs
• DeeperGCN: All You Need to Train Deeper GCNs
• Chip Placement with Deep Reinforcement Learning
• Neural Arithmetic Logic Units
• Inductive Representation Learning on Temporal Graphs