Search to Capture Long-range Dependency with Stacking GNNs for Graph Classification

Abstract

In recent years, Graph Neural Networks (GNNs) have been popular in the graph classification task. Currently, shallow GNNs are more common due to the well-known over-smoothing problem facing deeper GNNs. However, they are sub-optimal without utilizing the information from distant nodes, i.e., the long-range dependencies. The mainstream methods in the graph classification task can extract the long-range dependencies either by designing the pooling operations or incorporating the higher-order neighbors, while they have evident drawbacks by modifying the original graph structure, which may result in information loss in graph structure learning. In this paper, by justifying the smaller influence of the over-smoothing problem in the graph classification task, we evoke the importance of stacking-based GNNs and then employ them to capture the long-range dependencies without modifying the original graph structure. To achieve this, two design needs are given for stacking-based GNNs, i.e., sufficient model depth and adaptive skip-connection schemes. By transforming the two design needs into designing data-specific inter-layer connections, we propose a novel approach with the help of neural architecture search (NAS), which is dubbed LRGNN (Long-Range Graph Neural Networks). Extensive experiments on five datasets show that the proposed LRGNN can achieve the best performance, and obtained data-specific GNNs with different depth and skip-connection schemes, which can better capture the long-range dependencies.

References in corpus (18)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Neural Message Passing for Quantum Chemistry
• Representation Learning on Graphs with Jumping Knowledge Networks
• Hierarchical Graph Representation Learning with Differentiable Pooling
• Simple and Deep Graph Convolutional Networks
• Self-Attention Graph Pooling
• Design Space for Graph Neural Networks
• Representing Long-Range Context for Graph Neural Networks with Global Attention
• GraphNAS: Graph Neural Architecture Search with Reinforcement Learning
• Rethinking pooling in graph neural networks
• Pooling Architecture Search for Graph Classification
• Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective
• AutoGraph: Automated Graph Neural Network
• Long Range Graph Benchmark
• AutoGEL: An Automated Graph Neural Network with Explicit Link Information
• Automated Graph Machine Learning: Approaches, Libraries, Benchmarks and Directions
• Graph Property Prediction on Open Graph Benchmark: A Winning Solution by Graph Neural Architecture Search

Cited by in corpus (1)

• AutoHGNN: Robust and Efficient Neural Architecture Search for Hypergraph Neural Networks