Auto-GNN: Neural Architecture Search of Graph Neural Networks

Abstract

Graph neural networks (GNN) has been successfully applied to operate on the graph-structured data. Given a specific scenario, rich human expertise and tremendous laborious trials are usually required to identify a suitable GNN architecture. It is because the performance of a GNN architecture is significantly affected by the choice of graph convolution components, such as aggregate function and hidden dimension. Neural architecture search (NAS) has shown its potential in discovering effective deep architectures for learning tasks in image and language modeling. However, existing NAS algorithms cannot be directly applied to the GNN search problem. First, the search space of GNN is different from the ones in existing NAS work. Second, the representation learning capacity of GNN architecture changes obviously with slight architecture modifications. It affects the search efficiency of traditional search methods. Third, widely used techniques in NAS such as parameter sharing might become unstable in GNN. To bridge the gap, we propose the automated graph neural networks (AGNN) framework, which aims to find an optimal GNN architecture within a predefined search space. A reinforcement learning based controller is designed to greedily validate architectures via small steps. AGNN has a novel parameter sharing strategy that enables homogeneous architectures to share parameters, based on a carefully-designed homogeneity definition. Experiments on real-world benchmark datasets demonstrate that the GNN architecture identified by AGNN achieves the best performance, comparing with existing handcrafted models and tradistional search methods.

References in corpus (12)

• Neural Architecture Search with Reinforcement Learning
• Neural Architecture Search: A Survey
• DARTS: Differentiable Architecture Search
• Fast Graph Representation Learning with PyTorch Geometric
• Predicting multicellular function through multi-layer tissue networks
• Large-Scale Evolution of Image Classifiers
• Neural Architecture Optimization
• Efficient Architecture Search by Network Transformation
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• GraphNAS: Graph Neural Architecture Search with Reinforcement Learning
• AGAN: Towards Automated Design of Generative Adversarial Networks
• Reinforced Evolutionary Neural Architecture Search

Cited by in corpus (20)

• AutoML: A Survey of the State-of-the-Art
• Design Space for Graph Neural Networks
• Graph Neural Networks: Taxonomy, Advances and Trends
• Automated Machine Learning on Graphs: A Survey
• RoSGAS: Adaptive Social Bot Detection with Reinforced Self-Supervised GNN Architecture Search
• Dirichlet Energy Constrained Learning for Deep Graph Neural Networks
• MANAS: Multi-Scale and Multi-Level Neural Architecture Search for Low-Dose CT Denoising
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• Search to aggregate neighborhood for graph neural network
• Probabilistic Dual Network Architecture Search on Graphs
• Learned Low Precision Graph Neural Networks
• Evolutionary Architecture Search for Graph Neural Networks
• Rethinking Graph Neural Architecture Search from Message-passing
• Simplifying Architecture Search for Graph Neural Network
• Policy-GNN: Aggregation Optimization for Graph Neural Networks
• Graph Neural Network Architecture Search for Molecular Property Prediction
• Neural Architecture Search in Graph Neural Networks
• Network representation learning: A macro and micro view
• Edge-featured Graph Neural Architecture Search
• Automated Graph Learning via Population Based Self-Tuning GCN