DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks

Abstract

In this paper, we propose a novel framework to automatically utilize task-dependent semantic information which is encoded in heterogeneous information networks (HINs). Specifically, we search for a meta graph, which can capture more complex semantic relations than a meta path, to determine how graph neural networks (GNNs) propagate messages along different types of edges. We formalize the problem within the framework of neural architecture search (NAS) and then perform the search in a differentiable manner. We design an expressive search space in the form of a directed acyclic graph (DAG) to represent candidate meta graphs for a HIN, and we propose task-dependent type constraint to filter out those edge types along which message passing has no effect on the representations of nodes that are related to the downstream task. The size of the search space we define is huge, so we further propose a novel and efficient search algorithm to make the total search cost on a par with training a single GNN once. Compared with existing popular NAS algorithms, our proposed search algorithm improves the search efficiency. We conduct extensive experiments on different HINs and downstream tasks to evaluate our method, and experimental results show that our method can outperform state-of-the-art heterogeneous GNNs and also improves efficiency compared with those methods which can implicitly learn meta paths.

arXiv commentKDD 2021, Research Track; Add refs

References in corpus (16)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Neural Architecture Search with Reinforcement Learning
• Neural Message Passing for Quantum Chemistry
• Neural Architecture Search: A Survey
• DARTS: Differentiable Architecture Search
• Fast Graph Representation Learning with PyTorch Geometric
• MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding
• RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
• Machine Learning for Integrating Data in Biology and Medicine: Principles, Practice, and Opportunities
• Graph Transformer Networks
• Link Prediction Based on Graph Neural Networks
• SNAS: Stochastic Neural Architecture Search
• Design Space for Graph Neural Networks
• Search to aggregate neighborhood for graph neural network

Cited by in corpus (9)

• Knowledge Graph Reasoning with Relational Digraph
• Pooling Architecture Search for Graph Classification
• Space4HGNN: A Novel, Modularized and Reproducible Platform to Evaluate Heterogeneous Graph Neural Network
• Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective
• Enabling Homogeneous GNNs to Handle Heterogeneous Graphs via Relation Embedding
• Large Language Model-driven Meta-structure Discovery in Heterogeneous Information Network
• EvoPath: Evolutionary Meta-path Discovery with Large Language Models for Complex Heterogeneous Information Networks
• Graph Neural Architecture Search with GPT-4
• Bridging the Gap of AutoGraph between Academia and Industry: Analysing AutoGraph Challenge at KDD Cup 2020