Graph Transformer Networks

Abstract

Graph neural networks (GNNs) have been widely used in representation learning on graphs and achieved state-of-the-art performance in tasks such as node classification and link prediction. However, most existing GNNs are designed to learn node representations on the fixed and homogeneous graphs. The limitations especially become problematic when learning representations on a misspecified graph or a heterogeneous graph that consists of various types of nodes and edges. In this paper, we propose Graph Transformer Networks (GTNs) that are capable of generating new graph structures, which involve identifying useful connections between unconnected nodes on the original graph, while learning effective node representation on the new graphs in an end-to-end fashion. Graph Transformer layer, a core layer of GTNs, learns a soft selection of edge types and composite relations for generating useful multi-hop connections so-called meta-paths. Our experiments show that GTNs learn new graph structures, based on data and tasks without domain knowledge, and yield powerful node representation via convolution on the new graphs. Without domain-specific graph preprocessing, GTNs achieved the best performance in all three benchmark node classification tasks against the state-of-the-art methods that require pre-defined meta-paths from domain knowledge.

Neural Information Processing Systems (NeurIPS), 2019

Cited by in corpus (39)

• A Generalization of Transformer Networks to Graphs
• Boosting the Speed of Entity Alignment 10*: Dual Attention Matching Network with Normalized Hard Sample Mining
• Comprehensive evaluation of deep and graph learning on drug-drug interactions prediction
• Multiplex Heterogeneous Graph Convolutional Network
• Multi-Behavior Graph Neural Networks for Recommender System
• Generalization and Representational Limits of Graph Neural Networks
• Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks
• GraphiT: Encoding Graph Structure in Transformers
• Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queries
• Homophily-oriented Heterogeneous Graph Rewiring
• Space4HGNN: A Novel, Modularized and Reproducible Platform to Evaluate Heterogeneous Graph Neural Network
• Heterogeneous Graph Transformer
• Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads
• Higher-Order Attribute-Enhancing Heterogeneous Graph Neural Networks
• Group Identification via Transitional Hypergraph Convolution with Cross-view Self-supervised Learning
• HAWK: Rapid Android Malware Detection through Heterogeneous Graph Attention Networks
• Graph-Aware Transformer: Is Attention All Graphs Need?
• Hop-Hop Relation-aware Graph Neural Networks
• Self-supervised Auxiliary Learning for Graph Neural Networks via Meta-Learning
• Node Classification Meets Link Prediction on Knowledge Graphs
• Leveraging Meta-path Contexts for Classification in Heterogeneous Information Networks
• TransCamP: Graph Transformer for 6-DoF Camera Pose Estimation
• Uniting Heterogeneity, Inductiveness, and Efficiency for Graph Representation Learning
• Temporal Graph Network Embedding with Causal Anonymous Walks Representations
• GIPA: A General Information Propagation Algorithm for Graph Learning
• Livewired Neural Networks: Making Neurons That Fire Together Wire Together
• Meta-Path Learning for Multi-relational Graph Neural Networks
• Font Completion and Manipulation by Cycling Between Multi-Modality Representations
• Layer-stacked Attention for Heterogeneous Network Embedding
• Sentence Structure and Word Relationship Modeling for Emphasis Selection
• Generating the Graph Gestalt: Kernel-Regularized Graph Representation Learning
• Optimizing Graph Transformer Networks with Graph-based Techniques
• Neural PathSim for Inductive Similarity Search in Heterogeneous Information Networks
• Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning
• VSGM -- Enhance robot task understanding ability through visual semantic graph
• HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization
• Improving Expressivity of Graph Neural Networks
• On Inductive Biases for Machine Learning in Data Constrained Settings
• Sparse Graph to Sequence Learning for Vision Conditioned Long Textual Sequence Generation